Sunday, March 29, 2015

Next Steps

After the last effort I tried the following things:

First, I tried to stand-up my own Core Data stack by trying to piece together the code from MagicalRecord’s setup method.  I was doing fine until I ran into the code that set a flag in MagicalRecord to indicate that iCloud was being used.  It was a private method.  I need to look to see where that flag is used, if it turns out it is used inside MagicalRecord, then I will need to stand-up my own stack from the ground up.  I’m wondering if I could do that and then set the right final objects on MagicalRecord and it would work.  

I next deleted the app from my device.  In the end I lost all my stored records.  But the app did come back and I was again able to use it (at the current model version number I was on).  I still need to try a sane migration.  

I found out auto-migration is the only thing supported for Core Data iCloud sync.  But I still had this issue where my local stores were not syncing.  I ran across an article that talked about a method called a NSPersistentStoreCoordinator method, removeUbiquitousContentAndPersistentStoreAtURL.  Which the documentation says will delete all ubiquitous content for all peers for the persistent store at a given URL and also deletes the local store file.  (http://www.objc.io/issue-10/icloud-core-data.html)

So I set this up and on my first device as follows and ran it:

   class func resetCloudStore(completion: ((Void) -> Void)?) {
       let containerId = getCloudContainerId()

       let cloudURL = NSPersistentStore.MR_cloudURLForUbiqutiousContainer(containerId)
       var error: NSError? = nil
       let contentNameKey = getUbiquitousContentNameKey()
       let options = NSDictionary(objectsAndKeys: contentNameKey, NSPersistentStoreUbiquitousContentNameKey)
       let storeURL = NSPersistentStore.MR_urlForStoreName(MagicalRecord.defaultStoreName())
       MagicalRecord.cleanUp()

       let result = NSPersistentStoreCoordinator.removeUbiquitousContentAndPersistentStoreAtURL(storeURL, options: options, error: &error)
       NSLog("Finished deleting ubiquitous store with result \(result)")

   }

To make this work I created two helper methods:

getCloudContainerId() - returns a String of this form "iCloud.\(bundleIdentifier)"
getUbiquitousContentNameKey() - returns the same String we used when setting up MagicalRecord

I also added a completion block which I am not using right now but intend to later.

The result was the local store went away, so I lost all my local data.  I then went to my second device and did the same.  What happened next was unexpected.  Within a few seconds the first device got all of the data from the second device.  Syncing was now working again.  That isn’t what I expected.  I had expected that I would have had to enter my data all over again.  It seems somewhere the data was stored, got sync’d to the cloud account and then sync’d to all the attached devices.  

I’m now at a point where I want to be able to turn on and off iCloud syncing.  Actually, I only want to be able to turn it on (never to turn it off again).  Reading more online I am still trying to figure out how this works.  There could be a book written just on Core Data sync.  There probably should be one written.  I’d buy it if it was any good.  It would have to only cover the the new API though.  I feel like that is what the Ensembles guys are doing, just wish the book was more than 60% complete (as of March 28, 2015).  That would probably swing my decision to use their package.

At any rate, back to the issue at hand.  How do you do a one way transition to cloud syncing?  Here is what I think I know so far.  When the Core Data stack is stood up (I’m using MagicalRecord but I don’t think that really matters much here) there is a local “backing” store and a cloud store (the ubiquitous store).  Any time I save a NSManagedObject it goes to the local “backing” store and eventually is sync’d to the cloud store.  

When I do the same setup but only go to the local store (i.e. don’t add the icloud options) I get an empty local store.  My hunch was that I was getting two different stores for the local store depending on whether I had sync enabled or not.  After doing a good bit of debugging I confirmed that was the case as the URL’s for the local store and local “backing” store didn’t match between sync versus no-sync modes.  

In the same article I referenced above there is a section on how to turn iCloud sync on and off.  I have started down that road.  So far it’s not working.  It’s kind of interesting, for a long time I could not get Core Data iCloud sync-ing to work, now I can’t get it to stop!!  

Monday, March 23, 2015

Onward and Upward

Well when I last left off I was trying to verify step 3 of Apple’s iCloud Programming Guide For Core Data.  I failed miserably.  I am coming to the realization that the document is outdated.  When I removed the app from all my devices and the simulator and waited an hour, and enabled airplane mode the log still said Using Local store 0 and my data was still there.  I have no idea how this is supposed to work or why it isn’t working.

I decided to press onward and maybe something will make sense.  One change I did make is now instead of listening to the Core Data messages for didChange and willChange I switched to use MagicalRecord’s (kMagicalRecordPSCDidCompleteiCloudSetupNotification
and kMagicalRecordDidMergeChangesFromiCloudNotification)

Aggregations:
After getting the top level object to save I turned my attention to how would I manage a child object in an aggregation.  I used a tutorial I got from Ray Wenderlich’s site (http://www.raywenderlich.com).  The general steps are:
  • clone the collection from the parent object into a mutable collection
  • change the collection as desired
  • Set the collection back on the parent object
  • store the parent object

They also alluded to the fact open source projects tend to provide helper methods for this.  I couldn’t find anything looking through the MagicalRecord source so I rolled my own.  

I ran into a problem where during the third step I was having an inconsistent object failure.  It turned out this was how I was loading the parent object to begin with.  I was loading it on a separate thread and then returning it to the main thread.  The end result of all this was the MOC was not set on the parent object and trying to make a change to it later resulted in this error.  This seems like a design flaw in my code.  I need to rethink this.

What about Parse?

I was playing an iOS game last night and they had me sign-up.  I had two choices, enter my email or sign-in via facebook.  It got me to thinking about my experience with Parse.  I might could create my own login/signup page where all I asked for was the user’s email address.  Would that be enough to simulate the iCloud Core Data syncing without having them have to create a full fledged account(and remember their password)?  I need to think more on this one.

I ran across a project called “Sugar Record”  It looks promising, but I’m not willing to bite that one off until it matures more.

Retrieving NSDate from NSManagedObject

I found that in my NSManagedObject (generated by XCode) my date objects were declared as:

@NSManaged var dateEntered: NSDate

This seemed to trip Swift up when I tried to reference the dateEntered attribute (particularly if it was nil).  I ended up having to protect myself from this by using this code:

       if let _dateEntered = object.dateEntered as NSDate? {
           dateEnteredLabel.text = "\(_dateEntered)"
       }

Notice the explicit downcast to an optional NSDate. Kind of strange, but otherwise the compiler complained about the attribute not being able to be nil when clearly it can be.  I wonder if I could manually declare the NSDate attribute on the object as optional?

Here We Go Again

So I was thinking about my progress and I thought that I should try a migration just to make sure I was comfortable with how that would work.  Unfortunately, I started down that path before I knew it.  In making changes, I accidentally changed some attributes of my model without creating a new version.  That caused me to get a “Can’t find model for source store” when testing on my device.  After some investigation I realized that when MagicalRecord sets up the store it sets it up to be auto migrating.  This normally would be fine had I not messed the model up by making a change to it without first creating a new version.  My first attempt at fixing this was to create a new model version and set that as the default.  No luck.  I then poked around a bit and figured I would set MagicalRecord.shouldDeleteStoreOnModelMismatch to true, but that didn’t fix it either, so I was left with what do I do?

I looked around for a way to do a manual migration with MagicalRecord.  I even found a StackOverflow reference back in 2013 to an experimental method to setup the stack so that you could provide a MappingModel. But when I checked the latest code that didn’t seem to have made it in the latest release.  Sooo, I am left with thinking that I need to stand the stack up manually.  I’m not looking forward to that.

Yes I could probably delete the app from my device and reinstall, but what happens with the iCloud store? I still am unclear how the iCloud store would ever get migrated.  Oh well, more to learn I guess.

Sunday, March 15, 2015

Back To The Basics

I published my first app (Pain Logger) on the app store about two years ago.  For the app, I had some very simple data requirements. Essentially I wanted to store the user’s data in a database using Core Data and then sync that to the cloud so it would be available to other devices the user owned.  Seemed simple at the time, but in practice it wasn’t.  Like many other developers, I found that syncing wasn’t all it was cracked up to be.  

This post has a good history of what I and apparently everyone else was dealing with:


To get my app out I reluctantly decided to disable the syncing capability.  My goal was to add that feature back in, once I learned more.

Fast forward two years and this issue has continued to be on my radar.  I have read countless blogs and tried different solutions but none have quite given me what I was looking for.  

I had two events recently that have re-sparked my interest again in this. First, I had a customer contact me about why my app didn’t have that ability.  Second, I have a new app (written in Swift) that I have been working on for about three months now and it has the same basic database syncing requirements.

I first considered using Core Data on the client and CloudKit for the cloud support.  I looked into it enough to know that I didn’t like the idea of having to marshal my objects stored on the client device into name value pairs to be stored using CloudKit and then having to do the reverse when getting it back.  On the surface this looked like it would be a lot of code.

I next looked at using two third party packages, MagicalRecord and Ensembles.  I actually got them working.  The one thing that kept nagging at me was the amount of third party code I was using that abstracted my code away from the basic Core Data.  Not to mention Ensembles has a free for version 1.0 but "you must pay for version 2.0/source code/book/updates" pricing model.  That made me a little hesitant to say I really needed it.

About a month ago I started looking into using the Parse framework.  This seemed very promising and I was up very quickly with syncing to their cloud service.  However, my new app has a need to store data locally when not connected and I wanted to only enable the syncing capability if the user purchased that capability as an in-app purchase.

I started working with Parse’s anonymous user capability and local storage and it sort of worked, but it seemed weird to me in my use case that the user would pay for the in-app purchase to sync their device and then I would have to have them create an account (so it could be saved in Parse). 

I thought of various schemes of creating that user behind the scenes but I’m not convinced that would work. One great advantage I see in the Parse solution is you as a developer have great access to the user’s data stored in the cloud including the actual User login information, so if in the future there is a problem you might be able to fix it.  One downside of Parse was that it was free for the first tier but if the app usage started to pick up you could be forced to pay a monthly fee to keep up with the increased traffic.  I don’t think that is necessarily a deal killer just something I wasn’t sure about.  

One thing that had always nagged at me was in a session at WWDC last year one of the speakers alluded to how bad Core Data syncing was when it was first implemented (that was the time I was trying to get my first app out).  The speaker assured the audience that a lot of work had gone into fixing those issues.  

Sooo, long story short, I decided to relook at just using plain Core Data syncing to see if it really has been fixed.  However, I still want to use MagicalRecord as it seems to be a good compromise between raw Core Data and abstracting all of it away.  So I have embarked down this path.  Here is what I have done so far:

Step 1: Adding an iCloud Persistent Store to Core Data

I setup my new project to use Cocoapods and pulled in the MagicalRecord library and wired up the BridgingHeader as I am doing all my new projects in Swift to force me to learn the language.

I created my data model. One thing of note, if you let XCode build the NSManagedObject subclasses of your data model and you check the box “Use scalar properties for primitive data types” your Date attributes will be declared as NSTimeIntervals (which wasn’t what I wanted as I wanted dates to be nullable even though I had declared the attribute to be Optional)  Would be nice if you could do this on an individual attribute level.

Next was to begin setting up the Core Data stack using the documentation for MagicalRecord(MR) and comparing with the iCloud Core Data Programming Guide from Apple.  My first hurdle was what MR setup method should I use and how the parameters map to Apple’s setup instructions.  

I figured I needed to use the most configurable version of the MagicalRecord.setupCoreDataStackWithiCloudContainer method.

Unfortunately this method is not documented so I needed to figure out what each of the parameters are and how they map to the underlying Core Data implementation. I have come up with the following definitions for each:

  • containerID - This eventually gets mapped to bucketName in the MagicalRecord code. For iCloud support it must begin with “iCloud.” and I just appended my bundle identifier to that. It is used to create the URL for the ubiquity container.  In Apple’s documentation this is used to create the storeURL. MagicalRecord will create what it calls the cloudURL with this and then attach that to the NSPersistentStoreUbiquitousContentURLKey in the options dictionary.
  • contentNameKey - This is the name of the persistent store in the ubiquity container. If you use a name with periods you will get an error.  After googling this a bit and trying things out I found that if I just used {my app name}_DataStore that would work fine.  This value is used for the value to the NSPersistentStoreUbiquitousContentNameKey in Apple’s documentation.
  • localStoreName - This is the name of the database as stored on the user’s device. MagicalRecord will use this to create NSPersistentStore for the local database.
  • pathSubcomponent - This can be nil.  I presume this is used in cases where you might want to store different stores under the same top-level ubiquity container. Since my app was one database I didn’t need it.
  • completion - This is a completion block which is called after the default persistent store has been set.  Note, this appears to be called before the iCloud container is setup.

After setting all this up I ran my app and as is mentioned in Apple’s documentation (the first Checkpoint) I saw in the log the two log messages, first using the local store and then not using the local store.

Some other notes to this point.  
  • When setting up my entitlement it was important to let XCode create one for my app. Essentially what I did was turn on the iCloud capability ensure Key-value Storage and iCloud Documents were checked and choose to Use default container and ensured I had my app’s iCloud container (remember it must start with iCloud.) listed and checked in the list.
  • When testing this on the iOS Simulator I found I had to be logged into a valid iCloud account.  Yeah I know, that seems obvious, but you try a lot of things when debugging.

After all of this here is the code I ended up with for step one.

       var bundleIdentifier:String = ""
       if let infoDictionary:NSDictionary = NSBundle.mainBundle().infoDictionary {
           bundleIdentifier = infoDictionary.objectForKey(kCFBundleIdentifierKey) as String
       }
       let containerId = "iCloud.\(bundleIdentifier)"
       let contentNameKey = "GoalTrak_DataStore"
       MagicalRecord.setupCoreDataStackWithiCloudContainer(
           containerId,
           contentNameKey: contentNameKey,
           localStoreNamed: STORE_NAME,
           cloudStorePathComponent: nil,
           completion: {_ in
               NSLog("Finished setting up CoreData stack")
           })

Two notes about the code.  I chose to build the containerId by getting the bundle identifier from the mainBundle and appending “iCloud.” on the front of it.  I could of just hard coded it.
Also the STORE_NAME is just a class level constant for the name of the local database.

Step 1 complete!!

Step 2: Reacting to iCloud Events

This step just seemed to be adding a listener for when the NSPersistentStoreCoordinatorStoresDidChangeNotification was fired.  I added this line above the code I did in step one since the Apple documentation implied this should be done early.  

NSNotificationCenter.defaultCenter().addObserver(self, selector: "persistentStoreDidChange:", name: NSPersistentStoreCoordinatorStoresDidChangeNotification, object: nil)

Note: I did not listen to a specific NSPersistentStore as I wanted MagicalRecord to do as much of the heavy lifting as it could so at this point in the code I did not have a NSPersistentStore to listen to.

In the selector code I just printed out the notification’s userInfo:

   func persistentStoreDidChange(notification:NSNotification) {
       NSLog("Persistent store did change")
       if let _info: [NSObject:AnyObject] = notification.userInfo {
           for key in _info.keys {
               NSLog("   Key: \(key) Value: \(_info[key])")
           }
       }
   }

Step 2 complete!!

Step 3: iCloud Performs a One-Time Setup
At this point I realized that the Apple’s documentation is for a basic Core Data/iCloud setup and as such assumes you have direct access to the PersistentStoreCoordinator and uses the basic notifications.  MagicalRecord abstracts a good bit of that away and rightly so as there are now more events that are more specific to the Core Data stack that MagicalRecord has setup.  So I decided to refactor a bit and begin listening to some other more MagicalRecord specific events.  Also in this step I had to setup my first saving of a record.  This is accomplished with a simple algorithm of getting the ManagedObjectContext (MOC) for the object that needs to be saved and calling it’s MR_saveWithOptions method, passing in a completion block.

I know this is vague, but it is beginning to get app specific.  My goal here is to document the events that MagicalRecord will send as an object gets saved.

One thing to get MagicalRecord to play nicely with Swift is you have to tell MagicalRecord what the entity name is, otherwise you will see errors when saving, like “entityForName: could not locate an entity named ‘{project name}.{class name}’ To fix this I had to add a class method to each NSManagedObject instance that looks like this (assuming your class name was Foobar)

   class func MR_entityName() -> String {
       return "Foobar"
   }

You have to do this for each subclass of NSManagedObject that you have.  Additionally you have to go into your Model and set each of the Entities class to a fully qualified name.  So for instance using the example above and assuming your project was named MyProject you would enter MyProject.Foobar into the class attribute for the Foobar entity.

At this point I can create an object, and show a list of them.  One thing that isn’t working is when the app first stands up, my initial list of objects is not shown in my UITableViewController.  I think the problem here is the Core Data stack isn’t fully stood up before the viewDidLoad method in my initial view controller is called.  For now I just added a refresh button.  But I will add a notification this week and have my initial view controller listen and react to that.

That’s where I have stopped for now.  My next steps are to load the app on two devices and follow the checkpoint steps in Apple’s documentation.  I’m not convinced that I won’t need the Ensembles framework or I might even end up back using Parse.  We’ll see as I continue traveling down this road.  I think my main goal is to learn how this works, and have some type of service layer that I can apply to other apps as I build them.  We’ll see how that goes.

Monday, March 9, 2015

Blog Reboot

It's been a long time since I posted to this blog. I've decided that it is time to reboot this blog and share my experiences (no matter how mundane they may be). So what do I intend to talk about? My plans are to start or restart this blog as more of a journal about my professional journey as a programer. I intend to start first by just discussing the things I am working on right now particularly the highlights and low lights. I hope to post once a week. We'll see where this goes.

So first, who am I? I am a developer working for a large corporation in Raleigh NC. I have been a developer for 26 years and after a fairly significant bit of time in our nation's military I settled down to my current day job (that of programmer) about 16 years ago. I have worked on a bunch of different projects, at startups and now for the past ten years at my current employer (which is definitely not a startup). Up until the last year I primarily used Java, concentrating on the front end user interfaces in AWT and SWT (for you Eclipse gurus). About a year ago my company made a big push to switch from thick client applications to thin HTML5 based applications. I followed along and so for the past year I have been concentrating mostly on HTML5 clients and their integration with Java back ends.

Over my career I have used tons of different technologies. I started with C, quickly switched to C++ and then picked up Java. My "after work, work time" is spent on other areas of the development world I am interested in. Particularly mobile development on iOS. I have also dabbled in Unity, and Ruby. To date, I have released three apps to the various app stores. None very successful. My first was a simple Android game. When I found the tooling at the time to be fairly primitive and saw that iOS seemed to be a more lucrative endeavor I switched to iPhone/iPad development. That was about two years ago and since then I have released two iOS apps to the app store. One being a "light" version of the other.

One thing I struggled with early on was sharing the code base between these two apps. I feel with the module support now added to XCode, that problem may have been mitigated, but at this point I haven't had time to explore it much. I also have continued to develop my skills as a front-end web developer. Several years ago I was disgruntled with the state of J2EE applications and all the boiler plate junk you had to put together just to get a simple web site running with a backend including persistence. So I went in search of a simpler solution. I ended up deciding on Ruby on Rails. That has been a personally rewarding endeavor. As a result I have developed a few Ruby on Rails applications with dedicated front ends.

So where am I at today? This is what I intend to be the major thrust of this blog. I don't intend to talk much about my day job. My first iOS app required persistence on the device and I had hoped to sync that to the user iCloud account. This was when Core Data sync had just been released. It was a disaster. I found, like many other developers I have come to learn since, that the Core Data sync worked until it had conflicts and then the whole stack just came crashing down. You couldn't get updates and you had no idea what was going on. In reading other blogs I have learned this may have changed since then but I am still gun-shy over my experience.

So for a long time I had been looking at alternatives. Recently I have been trying out "Parse" and so far I like what I see. I have created a test app (written in Swift) and have successfully integrated Parse into it. Unfortunately, when I first implemented it I didn't include individual user support. So I tried to add that in and so far have fallen flat on my face. My idea is to create an anonymous user when the app starts up and allow the user to signup/login later in the use of the app. But it seems I get stuck when creating that anonymous user. Not sure why, the app just hangs and no information goes to the console. So that is where I am at today. Until next week.