Monday, May 25, 2015

Got It - Twice!!

Today is Memorial Day in the United States, and I just want to say THANK YOU, THANK YOU, THANK YOU, to all who have given everything for this country.  


Every year this time comes around, I try to think of ways to express to my friends what this day means.  I don’t think I do a very good job.  Having served for ten years in the Air Force myself, I worked with all types of people, but when it came down to “go time”, putting aside all personal desires, and working together as a team to accomplish the mission, there couldn’t have been a better group of people.  Again, I fail to do justice explaining what our folks in the  military go through an a daily basis.  But suffice it to say without them the way of life that we live each day would not exist and especially on this day I/we remember those who gave it all.  I fail to come up with words to express how grateful I am.   


Progress
I started this week with the goal of getting one of my persistent+sync test apps working.  It turned out I ended up getting two working (CloudKit and Ensembles). My plan in these blogs is to walk through the code of each.  This may take a while:  I’ll start first with the CloudKit solution as it was the first one I got working (although the Ensembles solution may be more interesting and fruitful in the end since that is probably what I will go with). I essentially used the design I talked about in my last post.


Here is how it works:


In the app delegate didFinishLaunchingWithOptions I added this code:


 let notificationSettings = UIUserNotificationSettings(forTypes: UIUserNotificationType.Alert | UIUserNotificationType.Badge | UIUserNotificationType.Sound , categories: nil)  
 application.registerUserNotificationSettings(notificationSettings)  
 application.registerForRemoteNotifications()  

This is boilerplate code to register for remote push notifications


I also implemented the application didReceiveRemoteNotifications callback method (the one without a completion handler) Here is the guts of that implementation:



     println("received remote notification: \(UIDevice.currentDevice().name)")  
     let note = CKNotification(fromRemoteNotificationDictionary: userInfo)  
     DataService.sharedInstance.handleNotification(note)  


Unlike other examples I found on the net, my solution uses custom zones, so I don’t need to get CKQueryNotifications, and rather can just take the userInfo as a simple CKNotification.  It gets passed to my DataService class that will do the sync’ing to the local store. Since I am using a custom zone the notification will be a ZoneChanged notification so I don’t need to setup a query.


If you remember from my last post I have the DataService which deals with CloudKit “stuff” and the LocalDataService which deals with the local client’s persistent store.  The client always uses the LocalDataService for its data needs. It never talks to the DataService, only the LocalDataService talks to it. So there is a basic assumption that the LocalDataService represents the most up-to-date data that the client has access to.  


So what does the DataService look like?  One thing I ran into early was the DataService is more closely connected to the data objects that it is syncing which is something I don’t like.  I’m still contemplating how to refactor this code so that it can be reused.


Another thing I realized was the DataService needs to have a way to tell the LocalDataService when things have changed in the cloud.  I chose to do this via a delegate pattern.  So when the LocalDataService stands up it registers as the delegate of the DataService.


Here is the DataService’s delegate protocol that the LocalDataService implments:



 protocol DataServiceDelegate {  
   func entitiesLoaded(entities: [LocalEntity])  
   func getLocalEntity(localRecordID: LocalRecordID) -> LocalEntity?  
   func entityAdded(entity: LocalEntity)  
   func entityUpdated(entity: LocalEntity)  
   func entityDeleted(localRecordID: LocalRecordID)  
   func hasEntityWithID(localRecordID: LocalRecordID) -> Bool  
   func handleRemoteError(error:NSError)  
   func databaseReset()  
 }  


A couple of things to note about this.  First, as I mentioned earlier,  you can see the DataService is very linked to the type of object’s it is persisting.  Also I chose to create a LocalEntity object.  This is the object that is used throughout the application. The DataService “marshals” the CKRecord’s it gets from the CloudKit database into and out of these objects.  The LocalDataService only uses these objects.  I’ll talk about this object structure in a future post.
  
Now onto a discussion of the DataService itself.


The DataService has the following instance fields:



   let container : CKContainer  
   let privateDB : CKDatabase  
   let subscriptionID = "subscription_entities"  
   let zoneID:CKRecordZoneID  
   var subscribed = false  
   var delegate: DataServiceDelegate?  


As you can see the DataService holds a reference to the CKContainer, the privateDB (as the records will not be shared among users), the zoneID and of course the delegate. Notice the subscriptionID is just a String.  I struggled with what this was supposed to look like (reverse DNS, UUID, etc).  It turns out it just needs to be a constant string that all clients using the app will use.


The subscribe field is initially set to “false” when the DataService is first initialized and then set to “true” when everything is setup.  This will be used to toggle sync’ing on and off and shortcut various DataService calls when sync’ing is not desired.


The DataService’s init method is as follows:



   override init() {  
     // Get the default container  
     container = CKContainer.defaultContainer()  
     // grab the private database  
     privateDB = container.privateCloudDatabase  
     // Create the zoneID  
     zoneID = CKRecordZoneID(zoneName: ZONE_NAME, ownerName: CKOwnerDefaultName)  
     // Call the super init  
     super.init()  
     // subscribe to the zone  
     connectToZone()  
   }  


This is a pretty standard init method.  Essentially I get the default container, and then it’s privateDB.  This is where I will store the user’s records.  Next I create a zoneID as I will be using a custom zone to store the records in.    


The only tricky part here is the ZONE_NAME constant.  This turns out to be essentially the default container name with the name of the zone tacked onto it.  So if your reverse DNS name was “com.companyA” and your app name was “myapp” and the zone was going to hold Entity objects then using this scheme the ZONE_NAME becomes “iCloud.com.companyA.myapp.entities”. I think it has to be prepended with “iCloud” as I have shown here since that is the default container id you will get when you turn on the iCloud capability.


The last part of initialization is connecting to the Zone.  Here is the connectToZone() method:



   private func connectToZone() {  
     // Fetch the record zone with the zoneID we are interested in  
     privateDB.fetchRecordZoneWithID(zoneID) {  
       recordZone, error in  
       if (error != nil) {  
         println("ERROR -- fetchRecordZoneWithID error code was: \(error.code)")  
         if (error.code == CKErrorCode.ZoneNotFound.rawValue) {  
           // Zone Not Found so we need to create it  
           dispatch_async(dispatch_get_main_queue()) {  
             self.createZone()  
           }  
         }  
       }  
       else {  
         println("zoneID exists")  
         dispatch_async(dispatch_get_main_queue()) {  
           self.subscribe()  
         }  
       }  
     }  
   }  


In this code I first attempt to fetch the zone I want to be subscribed to.  I am doing this to determine if the zone needs to be created.  It turns out that you only need to save the subscription to the container once.  I really couldn’t find anywhere on the net that explicitly said that, but through trial and error I found that was the case.  


If there was an error fetching the zone I look at the error code and if it is a ZoneNotFound error then I know I need to create the zone, if there isn’t an error then I still need to subscribe to the zone.  


This brings up another issue I ran into.  It turns out it is harder than I would like it to figure out what the error codes are.  When you get a CloudKit error and print it out all you see is a number.  Searching the net for CKError will give you the following link:

Direct link to CKError codes (Click the CKError link at the bottom of the page to open it)

You want to bookmark this you will be referring to it often.


Here is the createZone method:



   private func createZone() {  
     let zone = CKRecordZone(zoneName: ZONE_NAME)  
     privateDB.saveRecordZone(zone) {  
       recordZone, error in  
       if (error != nil) {  
         println("ERROR -- createZone error code was: \(error.code)")  
       }  
       else {  
         println("zone created!!")  
         dispatch_async(dispatch_get_main_queue()) {  
           self.subscribe()  
         }  
       }  
     }  
   }  


This is where we create the zone if it did not exist.  All I do here is created the zone and save it to the private database.  If no error occurs then I call the subscribe method which is what would be called if the previously listed connectToZone method found the zone already existed.


An important note here is that creating the zone and subscribing to the zone are one time actions that only need to be done by one of the user’s clients when connecting to CloudKit. Doing either of these a second time (say when the app is launched on a second device) results in an error being thrown. I don’t like seeing errors in my log so that is why this code is more complex than one would like.


On to the subscribe method:



   private func subscribe() {  
     if (subscribed) {  
       return  
     }  
     println("Subscribing to privateDB changes")  
     let subscription = CKSubscription(zoneID: zoneID, options: CKSubscriptionOptions.allZeros)  
     subscription.notificationInfo = CKNotificationInfo()  
     subscription.notificationInfo.alertBody = "notification from zone"  
     privateDB.saveSubscription(subscription) {  
       subscription, error in  
       if (error != nil) {  
         if (error.code == CKErrorCode.ServerRejectedRequest.rawValue) {  
           self.subscribed = true  
           println("already subscribed!")  
           self.listenForBecomeActive()  
           dispatch_async(dispatch_get_main_queue()) {  
             NSNotificationCenter.defaultCenter().postNotificationName(EVENT_SUBSCRIBED, object: nil)  
           }  
         }  
         else {  
           println("ERROR -- error subscribing: \(error.code)")  
           dispatch_async(dispatch_get_main_queue()) {  
             NSNotificationCenter.defaultCenter().postNotificationName(ERROR_SUBSCRIBING, object: nil)  
           }  
         }  
       }  
       else {  
         self.subscribed = true  
         println("subscribed!")  
         self.listenForBecomeActive()  
         dispatch_async(dispatch_get_main_queue()) {  
           NSNotificationCenter.defaultCenter().postNotificationName(EVENT_SUBSCRIBED, object: nil)  
         }  
       }  
     }  
   }  


First I check to see if the subscribed flag is set to “true” and if so bail as there is no need to run this code a second time.  


Next I create the subscription.  Since this is a custom zone subscription, it is easier to setup as I am interested in all changes to the zone.  I also add a CKNotificationInfo() object to the notificationInfo field of the subscription.  Even though Apple’s documentation for CKSubscription says that if you do not set the notificationInfo field push notifications will still be sent, I found that was not true.  One downside of doing this I found was that if the application is in the background and a push notification comes in the user will get a notification.  I’m not sure at this point how to “eat” these types of notifications.  I’ll have to look into that further.


Next the subscription is saved to the database.  If it gets rejected then I found that error code meant the subscription already existed so I set the subscribed flag to true and fire the notification that the DataService is now subscribed.  Otherwise I can’t deal with the error (more work is needed here) so I fire an event to say the subscription failed for some other reason and bail.


If saving the subscription worked then I do the same as I did if it got rejected.  At this point any listener to the EVENT_SUBSCRIBED event can now update its views.  I listen for this event in the first view controller that is stood up, so that it knows when to populate it’s tableView.


Whew!!, that was a lot, I think I’ll stop here for this week.  Next week I think I’ll go into some of the CRUD methods.


Reading
I’m one chapter from finishing reading Spring in Action.  I’m real excited about this last chapter as it is on Spring Boot and should springboard me into my next book “Spring Boot In Action”


News That Caught My Eye
Nothing of note this week.


Quick Looks
I actually dug hard into Ensembles documentation this week so that was my Quick Look effort but it turned out to be more of a project than a quick look.  I’ll have a post on this in the future.

Monday, May 11, 2015

Welcome to the Cloud

Progress
I spent this week converting my test app to CloudKit.  It was a success.  As long as the devices that are syncing are connected to the cloud account they sync fine.  I no longer had the missing records problem I had with the CoreData test app.

And more importantly, I understand what is going on and can verify (in the iCloud dashboard) the records are making it to the cloud.  I added code to resync when a device reconnects to the cloud.  Another success!  

But what about when a device is not connected to the cloud?  What then?  So I embarked down this path.  The idea is to store the results of the last cloud sync to the local device and if the device is not connected then use that local store.  Sounds a lot like the Parse solution, huh!?

So my current plan is to have two service classes.  The first one, called LocalDataService, serves the data to the app.  The app always goes to LocalDataService for any CRUD operations on the data it needs. After the LocalDataService is done with the operation it notifies the app via NSNotifications (so the app can refresh its views).  

The LocalDataService defines the types of notifications it will send, and the app code should register to listen for the notifications that are important to it.  

The LocalDataService also is the only interface to the second service class, the RemoteDataService class.  The LocalDataService is set as the delegate of the RemoteDataService.  The RemoteDataService calls the delegate’s methods when something in the cloud has happened.

In practice, this is how I am thinking this will work.  Let’s take an example where the app wants to add an object.  

First the app calls the LocalDataService with the object to be saved.  The LocalDataService stores the object in it’s local store.  For a new object (and this is important) it gives this object a temporary id. This is important so it can find it later.

The LocalDataStore then notifies the app that it has successfully saved the object (via NSNotification).  The LocalDataStore then passes the object to the RemoteDataStore and asks it to save the object to the cloud.  

Assuming the RemoteDataStore’s operation is successful, it calls its delegate (the LocalDataStore) to inform it, that the object was successfully stored in the cloud.  The LocalDataStore looks the object up in it’s cache (using the temporary id that was passed through) and replaces its local representation with the cloud stored version.  Finally the LocalDataStore notifies the app that the object was saved (via NSNotification).  Yes this is a two notification process, but the way I’m thinking about this is they are really two different operations.

This seems simple enough on the surface but in practice it is harder than you think.  Corner cases abound.  What if the user deletes a record while the device is not attached to the cloud?  

So far my only solution to this (and I’ve seen others use this) is to keep some type of status flag on the locally stored objects.  While I think this will work (I’m still in the implementation phase) it does seem like I will be writing a lot more code than I had hoped.

The advantages I see that make this a compelling effort are:
  • The LocalDataStore can be anything I want (NSDocument, CoreData, A NoSQL database, etc)
  • The RemoteDataStore can also be anything I want (CloudKit, Parse, etc). I would not see it being CoreData based as that is the problem I am trying to avoid.
 
The disadvantages I see are:

  • I keep thinking someone has already solved this problem and probably a lot better than I have
  • I’m not sure this solutions lends itself to a generic solution, more of a pattern I think. Subsequently, there may be a “boat-load” of code to write.

Reading
I’m now two thirds the way through Spring in Action.  I want to finish this up by the end of May and then launch into the Manning Spring Boot book.  I also am pretty jazzed about working on a Spring application.

News That Caught My Eye
This new Kickstarter $9 computer caught my eye this week. Imagine if they can make it that cheaply by next year.  A little more miniaturization and we can all carry one around in our pocket.

Quick Looks
I went over several articles explaining Realm.  That seems like the perfect companion to my sync design.  I wonder if this is the future of mobile databases?

Monday, May 4, 2015

Time To Pivot

Progress
I continued to work with my different syncing test apps this week.  In the end I ended up going over a couple of tutorials on CloudKit.  Coupling this with some conversations I have had online I have come to the following conclusions:

CoreData+Sync is very complex and there is a lot of “magic” going on.  It works great when it works, but when it fails you are stuck.  Adding a framework like Ensembles helps, but it too adds more complexity.

I don’t like complexity in my software designs.  It is tough to develop initially, and tougher to debug a year later when you have to support it (and have forgotten what you did the first time).

I need a simpler solution.  Something I can see from both ends, both on the client side and on the server side.

If I assume a mostly always connected device, then there are natural points in an app’s lifecycle where it would/should sync with a server (i.e. app initialization/shutdown, object saving/deleting/updating).

So I think for my new app I will abandon CoreData all together.  In going over CloudKit I found it fairly understandable.  There is a lot of code to write, but if I design it properly I’m hoping I will be able to abstract out the “CloudKity-ness” and separate my objects from that.  We’ll see how that goes.

I think this is the first steps towards an app with a custom server.  Maybe in the end that is what I will end up with.  At least going this route I should be able to make that migration fairly easily.  I figure I could also use Parse if I chose to go cross-platform.

What I am giving up in this approach is all the SQL-goodness that a real CoreData solution would provide.  

I will also have to write a lot of low level code (but I think it will be straightforward). I’m still a little concerned about what happens when I get conflicts during sync-ing, but I think the solutions to those are solvable.

My goal this week will be to get my test app running supporting CloudKit and see how that code looks.  Hopefully I’ll have more on this next week.

Reading
I continued reading Spring in Action.  My goal is to complete it by the end of May. No work on the test app though.

News that caught my eye

Microsoft had it’s developer’s conference this week.  I was amazed at the HoloLens demos.  If they can pull that off, then the applications for that technology are HUGE.  It occurs to me that Microsoft is acting like a company that is “hungry”.  With all the cash they have that could make them a formidable force in the computing industry once again.  It will be fun watching this play out.