In the real world, at some point, we all have to deal with data that is fetched from a backend service, page by page. It's called paging. Everybody does this! Nothing to be ashamed of ;-) Here is one of the ways in which a Swift pattern (well, not specific to Swift, but visible in Swift) can be used to build a neat solution that allows me to fetch data in chunks and display it in the application.

This approach involves Generators. A generator is a state machine that produces new data continuously. So instead of getting all the data at once, I can ask for chunks of data, calling the generator over and over to generate new data. For example, in Swift GeneratorType is a data source for SequenceType.

It's a very simple idea with a neat interface, yet it's powerful. It's all about asking for more data by calling next() - it advances to the next element and returns it, or nil if no next element exists.

generator.next()

GeneratorType protocol is defined as follows:

protocol GeneratorType {
    typealias Element
    mutating func next() -> Element?
}

... this is good for a sequence, but it's not good enough for asynchronous operations. That's why I created a similar Generator, but this one is ready to handle asynchronously generated data (like from network requests).

I'll demonstrate how to build a simple generator, which in conjunction with specialized functions can simply do the job. The job is: fetch data in chunks (pages).

Let's start.

The Generator

At first I defined AsyncGeneratorType protocol

protocol AsyncGeneratorType {
    typealias Element
    typealias Fetch
    mutating func next(fetchNextBatch: Fetch)
}

• Element describes the type of data to generate
• Fetch is a closure type, defines a type for data fetching function
• next(fetchNextBatch: Fetch) is the function to generate the next page of data

Now, let's build a generator that conforms to the AsyncGeneratorType protocol we've just defined, where:

• Element is an array of generic type T.
• Fetch is a closure with offset, limit and completion parameters. Completion is called after data is fetched successfully by the function - fetching may be an asynchronous operation.

    typealias Element = Array<T>
    typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void

Keep the current offset and page size limit:

    var offset:Int
    let limit: Int

    init(offset: Int = 0, limit: Int = 25) {
        self.offset = offset
        self.limit = limit
    }

Finally implement a next() function, the core of the paging:

    func next(fetchNextBatch: Fetch) {
        fetchNextBatch(offset: self.offset, limit: self.limit) { (items) in
            self.offset += items.count
        }
    }

The next() function gets a function (closure) as a parameter. That function will be called every time a new page is fetched from the underlying storage. Because fetching may be an asynchronous operation, after data is fetched, the completion handler has to be called with the resulting items. At this point the generator will update internal state and prepare to serve the next page.

Here's the code of the generator:

class PagingGenerator<T>: AsyncGeneratorType {
    typealias Element = Array<T>
    typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void
        
    var offset:Int
    let limit: Int
    
    init(startOffset: Int = 0, limit: Int = 25) {
        self.offset = startOffset
        self.limit = limit
    }
    
    func next(fetchNextBatch: Fetch) {
        fetchNextBatch(offset: offset, limit: limit) { [unowned self] (items) in
            self.offset += items.count
        }
    }
}

Generate pages of data

What I love about generators is their simple interface. Once defined - it just works. Once I instantiate a generator:

var paging = PagingGenerator<Contact>(startOffset: 0, limit: 25)

I can simply ask for the next page every time I need it:

paging.next(...)

In my case (paging) I need a function that defines actual data fetching. The example function fetchNextBatch() below:

1. Downloads data from the website
2. Updates a local list of results
3. Returns the items back to the completion closure

private func fetchNextBatch(offset: Int, limit: Int, completion: (Array<Contact>) -> Void) -> Void {
    if let remotelyFetched = downloadGithubUsers(offset) {
        self.contacts += remotelyFetched
        completion(remotelyFetched)
    }
}

This is a specialised function that is responsible only for getting the data and returning the result. From now on it's all I have to implement to make the paging work with data from different sources.

Every time I ask for the next page, the generator will fetch it, preprocess it and add the result to my local list. Requesting next page looks like this now:

paging.next(fetchNextBatch)

Specialised functions

Actually, it's not convenient to have fetchNextBatch() responsible for fetching AND updating the UI. Let me change the interface a little to separate these two things into specialised functions. First, update the protocol with an additional parameter onFinish, called after new data is fetched:

func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)? = nil) {
    fetchNextBatch(offset: self.offset, limit: self.limit) { (items) in
        onFinish?(items)
        self.offset += items.count
    }
}

Then I can use it to update my UI:

private func updateDataSource(items: Array<Contact>) {
    self.contacts += items
}

by calling it like this:

paging.next(fetchNextBatch, onFinish: updateDataSource)

Result

Here's a final version of PagingGenerator:

protocol AsyncGeneratorType {
    typealias Element
    typealias Fetch
    mutating func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)?)
}

/// Generator is the class because struct is captured in asynchronous operations so offset won't update.
class PagingGenerator<T>: AsyncGeneratorType {
    typealias Element = Array<T>
    typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void
    
    var offset:Int
    let limit: Int
    
    init(startOffset: Int = 0, limit: Int = 25) {
        self.offset = startOffset
        self.limit = limit
    }
    
    func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)? = nil) {
        fetchNextBatch(offset: offset, limit: limit) { [unowned self] (items) in
            onFinish?(items)
            self.offset += items.count
        }
    }
}

It's not really important which version of Swift this is. What I want to show is the way in which I can deal with data that is downloaded in chunks (pages), in a way that's appropriate for Swift structures.

I can use it to incrementally update a UITableView data source, like this:

class ViewController: UIViewController {
    @IBOutlet var tableView: UITableView!
    private var paging = PagingGenerator<Contact>(startOffset: 0, limit: 25)

    private var contacts = [Contact]() {
        didSet {
            tableView.reloadData()
        }
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        paging.next(fetchNextBatch, onFinish: updateDataSource) // first page
    }
}

//MARK: Paging

extension ViewController {
    private func fetchNextBatch(offset: Int, limit: Int, completion: (Array<Contact>) -> Void) -> Void {
        if let remotelyFetched = downloadGithubUsersPage(offset) {
            completion(remotelyFetched)
        }
    }
    
    private func updateDataSource(items: Array<Contact>) {
        self.contacts += items
    }
}

//MARK: UITableViewDataSource

extension ViewController: UITableViewDataSource {
    func tableView(tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        return contacts.count
    }
    
    func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
        let cell = UITableViewCell(style: UITableViewCellStyle.Default, reuseIdentifier: nil)
        let contact = contacts[indexPath.row]
        cell.textLabel?.text = "\(contact.firstName) \(contact.lastName)"
        return cell
    }
}

//MARK: UITableViewDelegate

extension ViewController: UITableViewDelegate {
    func tableView(tableView: UITableView, willDisplayCell cell: UITableViewCell, forRowAtIndexPath indexPath: NSIndexPath) {
        if indexPath.row == tableView.dataSource!.tableView(tableView, numberOfRowsInSection: indexPath.section) - 1 {
            paging.next(fetchNextBatch, onFinish: updateDataSource)
        }
    }
}

Recap

Generators are nice. Allows easily separate and specialise functions to do one thing right.

I have separated and specialised:

1. Function to fetch data
2. Function to update UI after new data arrive
3. State machine to get next page

Here is a repository with the source code: GeneratedResults-UITableView

PS. Cover photo: https://www.flickr.com/photos/horiavarlan/4332381194