Thursday, December 13, 2018

A (failed) attempt to recognise the postie with machine learning

I've tinkered with machine learning in the past using TensorFlow but wanted to try Apple's tools which are extremely simple to use by comparison.

We have a low quality webcam pointing out into the street in front of the house that writes a file to a NAS every time it detects motion near the letter box. I often check these images to see if the postie has come. The project's objective is to automate looking at the images and recognise the postie.

Following Apple's documentation, I created an Xcode storyboard and trained it with a folder containing two sub-folders named "Postie" and "No Postie".

The storyboard is just this:

import CreateMLUI

let builder = MLImageClassifierBuilder()

builder.showInLiveView()

And it puts up a nice little UX where you drag your training folder.


I did try all of the augmentations shown above, it took a lot longer but wasn't more accurate in my case.

After saving the model, I created a simple macOS application with an ImageWell in it. You drag in an image and it shows the most likely tag like this:


Here's the main bit of the code to look at the image and display the classification.

    @IBAction func droppedImage(_ sender: NSImageView) {
        if let image = sender.image {
            if let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) {
                let imageRequestHandler = VNImageRequestHandler(cgImage: cgImage, options: [:])
                do {
                    try imageRequestHandler.perform(self.requests)
                } catch {
                    print(error)
                }
            } else {
                print("Couldn't convert image")
            }
        }
    }
    
    private var requests = [VNRequest]()
    
    func setupVision() -> NSError? {
        // Setup Vision parts
        let error: NSError! = nil
        
        do {
            let visionModel = try VNCoreMLModel(for: PostieClassifier().model)
            let objectRecognition = VNCoreMLRequest(model: visionModel, completionHandler: { (request, error) in
                DispatchQueue.main.async(execute: {
                    // perform all the UI updates on the main queue
                    if let results = request.results {
                        let prediction = results[0] as! VNClassificationObservation
                        let classification = prediction.identifier
                        let confidence = prediction.confidence
                        self.predictionTextField.stringValue = "\(classification) \(confidence)"
                    }
                })
            })
            self.requests = [objectRecognition]
        } catch let error as NSError {
            print("Model loading went wrong: \(error)")
        }
        
        return error

    }

Not much code is there.

While this all works, in my case, recognition of the postie in my low quality webcam image is very unreliable. Testing did show this, to be fair.


I suspect the issue is that the postie is a very small part of the image and, due to weather and the position of the sun, the images do vary quite widely.

As mentioned above, I tried training with all of the augmentations available but accuracy didn't improve. Also I tried cropping training images to just include the postie.

In summary, I'm impressed with how super easy it is to create and use ML models on Apple's platforms but I have a great deal to learn about how to train a model to get good results.

Sunday, December 02, 2018

Beautiful Don Dorrigo Gazette

After visiting Dorrigo several times I finally remembered to drop in and purchase (for $1) a copy of the hot-metal typeset "Don Dorrigo and Guy Fawkes Advocate".

The paper has been published since January 8, 1910 and is the last known newspaper in Australia to still be printed this way.

The paper has no photographs and the small type is somewhat wonky but it has a charm of its own.

Mostly local advertising but with a few interesting stories. Click on the images here and view up close to really enjoy it.



There's a Wikipedia page and an ABC story with video showing the production process.

New van configuration working well

After re-building the inside of the van to replace the double bed with a single bed/couch facing a long bench with shelves, I've taken the opportunity to travel north for a few days and try living in it.

The new arrangement is much better than before. Along the way I stopped at very pleasant spots and could enjoy the view while still being able to make cups of coffee and access everything with ease.

The woodwork on the bench isn't great and if I do it again there are things I'd change but in the end it was built with straight timber in a curving van interior.

The sink and manual pump tap is fantastic and makes me feel much more at home when doing things like cleaning my teeth.

Another change is that I've mounted an antenna base on the roof rack and reception on 20m is pretty good although I do get noise from the solar charge controller and fridge.


Once again my destination was a farm stay in Eden near Dorrigo and it really is beautiful countryside. The morning view from the side of the van.


Lovely rivers are all over the place. (Click photos to enlarge).


I'm still figuring out where to put things and even in a "tiny house" it's amazing how things can be hard to find.


The bed, which is slats supporting a medium density single camp mattress from Clark Rubber is very comfortable and I quite like going to sleep when the sun goes down.


I've purchased a battery monitor with a current shunt that displays voltage, current and then figures out power use. This is being trialled in the shed and will be installed in the van in due course.