Plan A

The past few weeks I have been working on my new app. It is a stock tracking app and I ran into the issue of how to get live stock prices for free.

To be honest I would pay a small monthly fee for this capability, but the sites I went to that alluded to the fact they might have a service I could pay for didn't publish their prices. They just said "contact us".

I figured that meant I couldn't afford them.

So that sent me down the tried and heavily worn paths such as yahoo finance and screen scraping.

For my application, I need three bits of information, given the equity's symbol.

I needed:

• Title
• Description
• Latest price

In my use case, the user enters the symbol, and I then retrieve the name and description to allow them to confirm this is what they want to use.

Later in the workflow, at specific touch points in the app, I want to retrieve the latest prices for all the equities the user is tracking.

In the end I decided I would choose a hybrid approach.

Most of the solutions I found pointed at using yahoo finance for live stock data. The problem with that was I could not find a way to get the equity's description information.

The only way I could figure to get the description information was lookup the symbol on google and then screen scrape the information.

So my hybrid solution is to screen scrape the information when the user is searching for an equity to track and then once they are tracking it use yahoo finance to pull the live data.

One caveat before I go into the code: I realize that yahoo has been bought out and there is no guarantee the solution I have will work even in the near future. That's one of the reasons I chose to have two routes to the data. But for now this works pretty well.

For this post I wanted to dive into the integration I have for yahoo finance. In a future post I'll talk about the screen scraping approach.

Prerequisites for this solution are: Alamofire

To make this clean and isolated, I created a StockService class like so:

import Foundation
import Alamofire

/**
 A struct to hold a quote for an equity
 */
struct EquityQuote {
    var symbol:String = ""
    var price: Double = 0.0
}

/**
 The Stock Service provides functions to lookup stocks and retrieve their latest prices
 
 - notes:
   see https://developer.yahoo.com/yql/
 */
class StockService {

I created a struct to capture the loaded data and as the comment mentions I lean on yahoo's yql service

The idea is consuming code/classes will call the function on this service object whenever the quotes need to be retrieved. The details of how the service is found (fyi, it's not a singleton) is not germaine to this post. Maybe I should add that to my list of topics to blog about.

At any rate, just assume consumers can get to the service and call the getLatestQuotes method:

func getLatestQuotes(symbols:[String], completionHandler:@escaping (([EquityQuote]) -> Void)) {

The function takes an array of symbols (as strings) and a completion handler function that will return a list of EquityQuote objects.

There are a lot of old posts out there about how to do this, some use YQL and others point to yahoo.finance directly. After a bit of experimentation I came to the conclusion that the YQL solution was more correct for 2017.

Another thing that annoyed me was the url call has to be url encoded (i.e. spaces turned into their safe ASCII equivalents, etc). A lot of the examples you will find hard code the conversion in their strings.

To me this made it hard to look at and debug. So I chose to write as I would in the yql console and right before making url encode it.

So I defined constants for the things in the query that don't change. More specifically the host, the query prefix and the query suffix:

let QUOTE_QUERY_HOST = "http://query.yahooapis.com/v1/public/yql"
let QUOTE_QUERY_PREFIX = "?q=select * from yahoo.finance.quotes where symbol in ( "
let QUOTE_QUERY_SUFFIX = " )&format=json&env=store://datatables.org/alltableswithkeys&callback="

Next I convert the symbols array into a single string and then cobble together the entire url.

var symbolsString = symbols.reduce("") {text, value in "\(text)\"\(value)\", "}
symbolsString = symbolsString.truncate(by:2)

let finalQuery = "\(QUOTE_QUERY_PREFIX)\(symbolsString)\(QUOTE_QUERY_SUFFIX)"
let encodedQuery = finalQuery.urlEncode()
let urlToCall = "\(QUOTE_QUERY_HOST)\(encodedQuery)"

Note the call to finalQuery.urlEncode(). This is a function I put in a String extension to encode all the characters that need to be encoded to go across the wire. I won't include that here, but if anyone is interested let me know.

Now that I have a safe encoded url, it is time to make the call via Alamofire:

let request = Alamofire.request(urlToCall, method: .get, parameters: nil, encoding: JSONEncoding.default, headers:nil)
    request.responseJSON() { response in
        let result = response.result
        switch result {

As you can see it is a "get" call and I expect JSON back.

If I get a success then it's time to parse it. This part of the code is UGLY!! and I need to go back and use something like "swiftyJSON" to clean up all the checks. But, at the time, I was doing a lot of experimenting to get this to work and it was easier to just peel one layer off at a time.

case .success:
    if let value = result.value as? [String: Any] {
         if let query = value["query"] as? [String: Any] {
              if let results = query["results"] as? [String: Any]  {
                   if let quotes = results["quote"] as? [[String: Any]] {
                        var equityQuotes = [EquityQuote]()
                        for quote in quotes {
                            let symbol = quote["symbol"]
                            let lastTradePrice = quote["LastTradePriceOnly"]
                            if let symbol = symbol as? String, let priceText = lastTradePrice as? String, let price = Double(priceText) {
                                 let equityQuote = EquityQuote(symbol: symbol, price: price)
                                 equityQuotes.append(equityQuote)
                            }
                        }
                    completionHandler(equityQuotes)
                    return
                }
            }
        }
    }
    completionHandler([EquityQuote]())
case let .failure(error):
    print("\(error)")
}

What is going on here is I basically am diving into the returned JSON until I get to the "quote" dictionary. I then pull out the information I need to build an EquityQuote object and if I have them, I construct the object and throw it into the array I will return. Finally after processing all of that, I call the completion handler to notify the calling code we are done.

Things I still have to do is deal better with error conditions. It may be that I change the method signature to take a failure handler as well.

Note, if I get nothing from the call I still return an empty array to the caller. I'm not sure in what cases this would happen but it seemed like the right thing to do in order to continue the code flow.

I could have just returned an array of doubles, but I figured by returning these EquityQuote objects the calling code could inspect the symbol of each one and match it up with the symbols it cares about. Again, it seemed safer.

That's it, so far it is working pretty well. My next post will talk about the screen scraping approach I took. Till next time.