Tag Archives: Python

Cortana: Home Automation A.I. (Intro)

House Temps

We’ll dive more into this later, but for now here’s an old screenshot of our house temps dashboard.

Meet Cortana…

This post has been a long time coming… a few of my friends and people I’ve met in my travels have been asking a lot of questions about a little project of mine, but thus far I’ve only mentioned it in passing on Google+ and twitter. This post will largely be a teaser, an intro, but will be the first in a series on Home Automation (HA), specifically, my take on it. In this series, I’ll try to outline my background in HA, the current state of my HA, a few of my goals and the general scope and direction of my project. I’ll start with an overview of the architecture, hardware and software then follow up with a deep dive into specific components of my setup including the monitoring networks, the chatbot user interface, some of the logic in my software, problems I’ve run into, a couple areas where I wish I had more expertise and hardware I wish existed. We’ll see where we go from there.

First, let me introduce you to my project, I call it “Cortana”… ok, many of you will recognize the name from HALO. My wife and I are both HALO fans and when I was trying to decide on a persona for my project after pouring through lists of AI names in Sci-Fi past and present, Cortana was the obvious winner. More on the need for a persona in later posts. I also refer to it as an “AI”. My project is pretty smart and it does learn from past conditions and results, but sadly there is no high level ground breaking artificial intelligence work going on here. Everything that happens is programmed to happen based on various conditions, commands or other input, as I said, there is learning involved, but it’s highly programmed. I use the term “Automated Intelligence.”

A little context

It might help to have a little background and context. I’ve always been interested in home automation, it started with switching a light remotely, this is where many home automation projects start, and promptly end. But two years ago we bought our ‘dream’ home on the outskirts of town; three bedrooms upstairs, one downstairs (my photo studio), front room, living room, a large kitchen and dining room, sizable garden, large back yard, three car garage, the works. For us it’s a crazy big house and we love it… We live in southwest Idaho, which is mostly high desert. Two months out of the year it’s freakishly cold. Two months out of the year it’s freakishly hot. But for 6-8 months of the year it’s nice enough outside that we’ll have windows open at some point during the day/night. Typically, in spring and fall we’ll have the windows open 24/7 for a month or so. You’ll see how these points matter as we dive further into my HA implementation.

After buying the house, I started in on some basic home automation (lights and such) but some manual tasks started to annoy me and caused me to take a step back, one big driver for the scope of my project is energy efficiency. Thankfully we have plenty of electrons here in Idaho and they’re fairly cheap, but I had inefficient processes/systems, so it ranks pretty high. We’ll dive deeper into my Home Automation Manifesto in the next post. But as my goals started to grow and the complexity of what I wanted to manage grew, I realized what I was whiteboarding was far different than your average HA project.

Off the Shelf?

There are a ton of off the shelf home automation systems out there. The vast majority of them suck. Ok, some of them are pretty impressive, but impressive carries a large price tag. And actually, even the really cool, crazy expensive systems don’t do some of the core requirements of my system. After searching various commercial systems, I knew it would have to be DIY, of course that’d make it much more fun too. I also spent a lot of time looking into the existing Open Source HA platforms, none of them quite fit what I was looking for. The better ones hadn’t been touched in ages and showed no signs of being resurrected. I wasn’t about to pull one out of the grave that was written in (insert any non python language here). So apologies in advance for not using (insert your favorite HA platform here). Generally the problems were 1) to simple 2) to complex for what they did 3) not extensible 4) horribly documented 5) Written in perl 6) Written in java 7) bad architecture 8) w. t. f. were you thinking?

That’s not to say that my project is the most amazing thing since sliced bread, but it works for me. Hopefully though, even if you don’t ever use a single line of code I’ve written, my thoughts, trials and errors as noted in this series will help you design and build your HA system.

Got Source?

Those that know me, or see my twitter bio, or have ever talked to me for more than 5 minutes, know that I’m an open source junkie. I love the open source community and what we are able to accomplish. Currently, my project is not open source, it will be one day. Once a component of it reaches some level of maturity, I will release it. Take WeatherAlerts for instance it’s part of my HA system and I have packaged it and released it via PyPI with the source is on GitHub. There are a couple other packages and snippets I have released to the wild and I have plans on releasing more over the next few months. It’ll probably be later this year before any core code is released. The reason I haven’t, is largely due to the fact that my core code is currently VERY specific to my implementation. 

Alright, enough rambling for now, I’ll try to post the first 3-4 parts of this every over the next 3-4 weeks, after that I’ll start into some deeper technical details and such every couple of weeks or so or as I add new components. I hope this series gives you some ideas. Please let me know if you have any questions, comments, ideas or such.

Posted in Tech Also tagged , , , , , , , |

WeatherAlerts Use Metrics

A couple years ago, I was needing a python script that would grab realtime Severe Weather Alerts and integrate that data into my Home Automation project. I couldn’t find anything (in any non-horrible programming language) that did what I wanted. So, I wrote a script to do it. It stayed hidden in the depths of my home automation project for several months until an acquaintance asked for a copy. I decided to go ahead and open source script, throwing it in a public github repo and packaging it for inclusion in PyPI. It’s called “WeatherAlerts.”  

Let me be completely clear, this package was a mess. It was my first foray into python packaging and it showed. It was also written much earlier in my python days, but, it worked (mostly). I decided that since it worked, if someone else could find use for it, or at least give them a starting point toward making something better… in the spirit of Open Source, I’d throw it out there. It’s had a few improvements over the past two years and recently it’s gotten much better but it’s still a functional mess though I’m working to fix that (the mess part).

Last year, my WeatherAlerts python package had a total of 3,574 downloads on PyPI. This year, it’s already at 2,409 with 1,287 so far this month… Since these numbers are just PyPI download counts, I don’t have any analytics on which to ponder, which drives me nuts. I do have analytics on the documentation I’ve posted and it has show a similar uptick recently. But it’s rather difficult to equate downloads to actual users considering bot downloads, redownloads, installs that are never used, cosmic events, etc. I’d love to know how many people actually use it…. When I decided to open source it, I assumed it might help as many as four or five users 🙂  Even if the PyPI numbers are way off, it is still being looked at a lot more than I would have expected.

Over the past few months, I’ve had a number of emails and social media contacts from users which has resulted in some great feedback and conversations. This makes me want to spend a few minutes some day to clean it up and make it presentable. 🙂 Regardless it has fulfilled the goal I had when I decided to open source it, that being to help more than 0 people, in the process I’ve learned more about python packaging, sphinx documentation and python as a whole.  

I have four public packages on PyPI and WeatherAlerts isn’t the most popular. NagParser is a package a Friend/Coworker and I open sourced shortly after I released WeatherAlerts. I just pulled the metrics for NagParser and it had over 17,300 downloads last year and is already sitting at 8,117 this year. The other two packages I have on PyPI are barely touched and getting less popular by the day (if that’s even possible) but I’m not offended, they’re very niche packages that are hardware or API subscription specific. 

I also have run my own PyPI/pip compatible server at home, for stuff that’s not quite open source ready or stuff very custom to me.

Posted in Tech Also tagged , , , |

Python Continuous Integration and Deployment (with Jenkins)

I’ve only been in the python community for about a year, so I’m still new, but I really do love coding in python. Anyway, I decided to open source a library (‘WeatherAlerts’) I started writing a few weeks ago.  I’ll post about it at some point in the future….  At first it was just a code repo on github, but after a bit of time working with it I wanted to package it and put it on pypi so it could be installed via easy_install or pip. With the state of change that python is in right now figuring out how to write the installer, have it support both python 2 and python 3 and get the package available on pypi wasn’t trivial. But now that it’s behind me, I have a much better understanding of python package management.

I had a simple test script that I was manually running every few commits and before pushing any changes out to the public repo. It worked, but being a Linux Systems Engineer though, this wasn’t going to stay that way long term. A lot of what I do in my day job is systems automation. Deploy servers, mange changes to large clusters, Continuous Integration and Continuous Delivery at the system level… So while my simple little python project was perfectly fine with being a manual build and deploy process, I couldn’t stand it.

So, I searched out a pythonic way to automate these tasks.

Continue reading »

Posted in Python, Tech Also tagged , , |