Posts in open source

How to Abstract Your Persistence Layer and Migrate to Another One on iOS With JustPersist

The original post is published on the JUST EAT tech blog at the following URL http://tech.just-eat.com/2017/03/02/how-to-abstract-your-persistence-layer-and-migrate-to-another-one-on-ios-with-justpersist/

JustPersist Banner

In this blog post we introduce a solution to deal with data persistence. We developed it for the Just Eat iOS app and we call it JustPersist. It’s available open source on Github at github.com/justeat/JustPersist.

JustPersist aims to be the easiest and safest way to do persistence on iOS with Core Data support out of the box. It also allows you to migrate to any new persistence framework with minimal effort.

I highly suggest to read my previous article The Easiest Core Data as the underlying concepts are explained there.

The main author behind JustPersist and its design is Keith Moon. Major kudos to Keith for the excellent execution in Swift!

Overview

At Just Eat, we persist a variety of data in the iOS app. In 2014 we decided to use MagicalRecord as a wrapper on top of Core Data but over time the numerous problems and fundamental thread-safety issues, arose. In 2017, MagicalRecord is not supported anymore and new solutions look more appealing. We decided to adopt Skopelos: a much younger and lightweight Core Data stack, with a simpler design, developed by Alberto De Bortoli, one of our engineers. The design of the persistence layer interface gets inspiration from Skopelos as well, and we invite the reader to take a look at its documentation.

The main problem in adopting a new persistence solution is migrating to it. It is rarely easy, especially if the legacy codebase doesn’t hide the adopted framework (in our case MagicalRecord) but rather spread it around in view controllers, managers, helper classes, categories and sometimes views. Ultimately, in the case of Core Data, there is a single persistent store and this is enough to make impossible to move access across “one at a time”. There can only be one active persistence solution at a time.

We believe this is a very common problem, especially in the mobile world. We created JustPersist for this precise reason and to ease the migration process.

At the end of the day, JustPersist is two things:

  • A persistence layer with a clear and simple interface to do transactional readings and writings (Skopelos-style)
  • A solution to migrate from one persistence layer to another with (we believe) the minimum possible effort

JustPersist aims to be the easiest and safest way for persistence on iOS. It supports Core Data out of the box and can be extended to transparently support other frameworks. Since moving from MagicalRecord to Skopelos, we provide available wrappers for these two frameworks.

The tone of JustPersist is very much Core Data-oriented but it enables you to migrate to any other persistence framework if a custom data store (wrapper) is implemented (in-memory, key-value store, even Realm if you are brave enough).

JustPersist is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod "JustPersist/Skopelos"
# or
pod "JustPersist/MagicalRecord"

Using only pod JustPersist will add the core pod with no subspecs and you’ll have to implement your own wrapper to use the it. If you intend to extend JustPersist to support other frameworks, we suggest creating a subspec.

Usage of the persistence layer

To perform operation you need a data store, which you can setup like this (or see related paragraph paragraph):

let dataStore = SkopelosDataStore(sqliteStack: <modelURL>)
// or
let dataStore = MagicalRecordDataStore()

Before using the data store for the first time, you must call setup() on it, and possibly tearDown() when you are completely done with it.

We suggest setting up the stack at app startup time, in the applicationDidFinishLaunchingWithOptions method in the AppDelegate and to tear it down at the end of the life cycle of your entire app, when resetting the state of the app (if you provide support to do so) or in the tearDown method of your unit tests suite.

To hide the underlying persistence framework used, JustPersist provides things that conform to DataStoreItem and MutableDataStoreItem, rather than the CoreData specific NSManagedObject. These protocols provide access to properties using objectForKey and setObject:forKey: methods.

In the case of Core Data, JustPersist provides an extension to NSManagedObject to make it conforming to MutableDataStoreItem.

Readings and writings

The separation between readings and writings is the foundation of JustPersist. Reading are always synchronous by design:

dataStore.read { (accessor) in
  ...
}

While writings can be both synchronous or asynchronous:

dataStore.writeSync { (accessor) in
  ...
}

dataStore.writeAsync { (accessor) in
  ...
}

The accessor provided by the blocks can be a read one (DataStoreReadAccessor) or a read/write one (DataStoreReadWriteAccessor). Read accessors allow you to do read operations such as:

func items(forRequest request: DataStoreRequest) -> [DataStoreItem]
func firstItem(forRequest request: DataStoreRequest) -> DataStoreItem?
func countItems(forRequest request: DataStoreRequest) -> Int

While the read/write ones allow you to perform a complete set of CRUD operations:

func mutableItems(forRequest request: DataStoreRequest) -> [MutableDataStoreItem]
func firstMutableItem(forRequest request: DataStoreRequest) -> MutableDataStoreItem?
func createItem(ofMutableType itemType: MutableDataStoreItem.Type) -> MutableDataStoreItem?
func insert(_ item: MutableDataStoreItem) -> Bool
func delete(item: MutableDataStoreItem) -> Bool
func deleteAllItems(ofMutableType itemType: MutableDataStoreItem.Type) -> Bool
func mutableVersion(ofItem item: DataStoreItem) -> MutableDataStoreItem?

To perform an operation you might need a DataStoreRequest which can be customized with itemType, an NSPredicate, an array of NSSortDescriptor, offset and limit. Think of it as the corresponding Core Data’s NSFetchRequest.

Here are some complete examples:

dataStore.read { (accessor) in
  let request = DataStoreRequest(itemType: Restaurant.self)
  let count = accessor.countItems(forRequest: request)
}

dataStore.read { (accessor) in
  let request = DataStoreRequest(itemType: Restaurant.self)
  request.setFilter(whereAttribute: "name", equalsValue: <some_name>)
  guard let restaurant = accessor.firstItem(forRequest: request) as? Restaurant else { return }
  ...
}

dataStore.writeSync { (accessor) in
  let restaurant = accessor.createItem(ofMutableType: Restaurant.self) as! Restaurant
  restaurant.name = <some_name>
  ...
  let wasDeleted = accessor.delete(item: restaurant)
}

written in core data, database, ios, magicalrecord, open source, persistence, skopelos Read on →

A Better Local and Remote Logging on iOS With JustLog

The original post is published on the JUST EAT tech blog at the following URL http://tech.just-eat.com/2017/01/18/a-better-local-and-remote-logging-on-ios-with-justlog/

JustLog Banner

In this blog post we introduce the solution for local and remote logging we developed for the Just Eat iOS app. It’s named JustLog and it’s available open source on Github at https://github.com/justeat/JustLog.

Overview

At Just Eat, logging and monitoring are fundamental parts of our job as engineers. Whether you are a back-end engineer or a front-end one, you’ll often find yourself in the situation where understanding how your software behaves in production is important, if not critical. The ELK stack for real-time logging has gained great adoption over recent years, mainly in the back-end world where multiple microservices often interact with each other.

In the mobile world, the common approach to investigating issues is gathering logs from devices or trying to reproduce the issue by following a sequence of reported steps. Mobile developers are mostly familiar with tools such as Google Analytics or Fabric.io but they are tracking systems, not fully fledged logging solutions.

We believe tracking is different in nature from logging and that mobile apps should take advantage of ELK too in order to take their monitoring and analysis to another level. Remote logging the right set of information could provide valuable information that would be difficult to gather otherwise, unveil unexpected behaviours and bugs, and even if the data was properly anonymized, identify the sequences of actions of singular users.

JustLog takes logging on iOS to the next level. It supports console, file and remote Logstash logging via TCP socket out of the box. You can also setup JustLog to use logz.io with no effort. JustLog relies on CocoaAsyncSocket and SwiftyBeaver, exposes a simple swifty API but it also plays just fine with Objective-C.

JustLog sets the focus on remote logging, but fully covers the basic needs of local console and file logging.

Usage

JustLog, is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod "JustLog"

Import it into your files like so:

// swift
import JustLog

// Objective-C
@import JustLog;

This logging system strongly relies on SwiftyBeaver. We decided to adopt SwiftyBeaver due to the following reasons:

  • good and extensible design
  • ability to upload logs to the cloud
  • macOS app to analyze logs

A log can be of one of 5 different types, to be used according to the specific need. A reasonable adopted convention on mobile could be the following:

  • 📣 verbose: Use to trace the code, trying to find one part of a function specifically, sort of debugging with extensive information.
  • 📝 debug: Information that is helpful to developers to diagnose an issue.
  • ℹ️ info: Generally useful information to log (service start/stop, configuration assumptions, etc). Info to always have available but usually don’t care about under normal circumstances. Out-of-the-box config level.
  • ⚠️ warning: Anything that can potentially cause application oddities but an automatic recovery is possible (such as retrying an operation, missing data, etc.)
  • ☠️ error: Any error which is fatal to the operation, but not the service or application (can’t open a required file, missing data, etc.). These errors will force user intervention. These are usually reserved for failed API calls, missing services, etc.

When using JustLog, the only object to interact with is the shared instance of the Logger class, which supports 3 destinations:

  • sync writing to Console (custom destination)
  • sync writing to File (custom destination)
  • async sending logs to Logstash (usually part of an ELK stack)

Following is a code sample to configure and setup the Logger. It should be done at app startup time, in the applicationDidFinishLaunchingWithOptions method in the AppDelegate.

let logger = Logger.shared

// file destination
logger.logFilename = "justeat-demo.log"

// logstash destination
logger.logstashHost = "my.logstash.endpoint.com"
logger.logstashPort = 3515
logger.logstashTimeout = 5
logger.logLogstashSocketActivity = true

// default info
logger.defaultUserInfo = ["app": "my iOS App",
                          "environment": "production",
                          "tenant": "UK",
                          "sessionID": someSessionID]
logger.setup()

The defaultUserInfo dictionary contains a set of basic information to add to every log.

The Logger class exposes 5 functions for the different types of logs. The only required parameter is the message, optional error and userInfo can be provided. Here are some examples of sending logs to JustLog:

Logger.shared.verbose("not so important")
Logger.shared.debug("something to debug")
Logger.shared.info("a nice information", userInfo: ["some key": "some extra info"])
Logger.shared.warning("oh no, that won’t be good", userInfo: ["some key": "some extra info"])
Logger.shared.error("ouch, an error did occur!", error: someError, userInfo: ["some key": "some extra info"])

It plays nicely with Objective-C too:

[Logger.shared debug_objc:@"some message"];
[Logger.shared info_objc:@"some message" userInfo:someUserInfo];
[Logger.shared error_objc:@"some message" error:someError];
[Logger.shared error_objc:@"some message" error:someError userInfo:someUserInfo];

written in elk, ios, logging, logstash, open source Read on →

From the Eyes of an iOS Dev at JUST EAT

It has been almost 6 months since my last blog post. Things have changed quite a lot since then. Six months ago I was still excited about my travel to San Francisco for the WWDC 2014, my girlfriend still had to move from Italy to London with me and definitely I wasn’t planning to switch job again any time soon.

Overture

I’ve been attracted by JUST EAT as a company since March 2014 but at that time it was too early for me to consider to change job. I met Ben Chester (the tech lead of the iOS team) when he gave a talk at Badoo offices (when I was still working there) and that evening he blew my mind. Later, I had a few chances to have a chat with the passionated guy he is and I immediately thought “Damn! I want to work with this guy, with brilliant guys like him and I want to work at JUST EAT!”.

Since then, I heard people talking extremely good about JUST EAT as a job place because of the values, the work environment, the company culture and the engineeristic approach to things. Every time I started with “Do you know JUST EAT as a company?” the answer was something like “Oh yeah! They are freaking cool! I have a friend working there, they do amazing stuff and he’s very very happy!”. They definitely were all good signs. Signs I decided not to underestimate anymore these days.

Another good sign was also the exposure and lots of information that the company promotes online with its tech blog giving a good insight of the technologies used, the people and teams working there and a good description of the Engineering. Benefits are also compelling.

JUST EAT offices hosted NSLondon a few times. Meetups, you know, are the perfect occasions to reach out the developers' community. I noticed too many cool companies failing at this.

After months of interest about JUST EAT and thoughts spinning in my head, I said to myself “let’s see if I have what it takes”. I decided to apply for the Senior iOS role in later October 2014 kicking off the process taking the test task. I joined the Consumer iOS app team at the begin of 2015 and after 2 months of excitement I’m summarizing some thoughts here.

wall

written in github, ios, just eat, london, open source, work Read on →

Working on Tasks With an Eye on Open Source Contributions

Have you ever realized that developers are never happy with the legacy code?

The definition of “legacy code” may vary:

  • code inherited from the previous developers of your company
  • code that doesn’t have test suites
  • code that is older that 10 minutes (…)

We all rarely find good code when joining a company, weird uh? Some reasons why good code is so hard to find can be:

  • the developer that worked on the code was good but couldn’t care less about doing things properly
  • the developer that worked on the code was simply unexperienced and created bizzare things
  • the developer that worked on the code was terrible at architecture design
  • too many developers worked on the same code without understanding what was already been done
  • code was not developed using the black box approach and without reusability in mind

All points but last are summarized here:

developers

written in github, ios, objective-c, open source, work Read on →