This is when i realize i may be working on too many projects… A few months (ok a lot of months) ago i started rewriting from scratch and greatly expanded the HydroDuino project’s software side, and had a working, if basic version,but completely lacking in modularity (basically it was a completely monolithic, tightly coupled app)  and hard to maintain and expand.

So i started an almost complete rewrite based on what was working and what wasn’t, aiming for a highly modular, yet lightweight system, and now things are progressing at full speed , so it was time for a “public anouncement” of the state of the project, since i still get quite a lot of emails of people wanting to use it/participate.

Foreword:

This article is meant to represent the current state of the project, and since its mean to be usable (hopefully) usable, and community driven, i would really apreciate any feedback ! If you think the whole thing, ore certain aspects of it are absurd, or unusable, or badly designed , or anything, don’t hesitate to contact me or leave a comment!

When the first version is released it will be available here : http://code.google.com/p/hydroduino/

Currently implemented features

It is now 3 semi seperate projects :

  • The core of the system , the “server” side backend is where most of the “magic” happens, and which provides all the main functionallity, the handling of sensors , hardware etc
  • The basic web based user interface frontend: in ajax with a python webserver backend that communicates with the core’s wsgi api
  • An android based user interface frontend (still in the very early stages)

General structure:

For a better understanding here is what the different things mean:

(Hardware connector is the same here as hardware node, for more explanation , see below in the “Where it is headed” part)

  • each environment is kept as separate as possible to avoid the problems of one of them affecting all of them : so each environment has its own sqllite db , its own scheduler(s), data access system etc
  • currently all the sensors are within their parent “HardwareNode” object and have a rather strong coupling with it : for example for an Arduino and its sensors, there is a “ArduinoNode” which is a child class of the general node class, that manages the connection to the arduino via serial  etc
  • the arduino side software is now a library + main code that alows the system (and you) to reconfigure the different ports, set triggers and automation etc

Plugin based system:

There are two main types of “plugins”

Hardware Types:
  • out of the box , Hydroduino will come with an “Arduino Node” : all you need to just plug in an arduino , configure the software, and have it running in just a few minutes
  • anyone can easilly add support for different type of hardware : ie Phydgets, Webcams etc with very little work , then just drag and drop the new code into the hardware plugins’ directory and then be able to use this new hardware through the generic interface
  • Converters:

    if , like many people , you have been tinkering around with Arduinos, electronics , sensors etc, you know that it is not always easy to read through a specific sensor’s technical sheet to be able to convert raw analog/digital values into usable data. Well the “converter” type plugins are there to make life easier : they will be a user generated, shared library of conversion functions, so that once someone has figured out a good conversion formula for a specific type of sensor, anybody else can just use that code automatically!  When you add a new sensor , just set its model name (the name of the actual physical sensor) , and if that converter exists, it will automatically be used when retrieving the data.

    Automation:

    • Schedules: you can set when you want to retrieve the information from a specific sensor, or when and how often you want a pump/motor/lamp whatever to run
    • Triggers: you don’t want the pump watering your plants to keep on working when there’s already too much water ? Just set a trigger on a water probe/level sensor to have it shut down automatically when certain conditions are met

    A small note about this: for now automation runs on the python host side, but i plan to have it be settable : if you want some time critical triggers etc, they will also be settable directly on the Arduino side of things , so there is no delay.

    Lightweight:

    • Even if it does complicate things a bit, the external dependencies of the python code are kept to a strict minimum, and i generally avoid big “framework” type dependencies (ie, no Django , etc required)
    • Small memory usage: the tests i have done for now of what an average user might use/have, generally come with a less than 4 Mb memory consumption, although this is not set in stone, as the software is still not as feature rich as i would like, and in a pre-alpha stage.

    Where it is headed

    While a lot of these features are currently implemented, and the general structure and how it works is much cleaner than before , it is far from finished, and in constant evolution (but getting there). These are a few of the problems i am facing:

    Lots of work left :

    • While the core of the system is functional , a lot of parts still are not !
    • a great deal of functions are not implemented yet
    • each major change in the basic systems required quite a lot of rewriting (since some of them are not set in stone 100% yet)
    • there currently is no UI to work with , since the underlying server’s structure has been completely altered
    • the project is way bigger than it was

    To depend or not to depend:

    Since i want to keep things lightweight , i am always a bit reluctant to add additional dependencies on other python packages/libraries/frameworks, but always reinventing the wheel is not a viable option either! This applies to :

    • ORM : (object relationship mapping): i have given small test runs off different Python Orm (Autumn, Storm ,SqlAlchemy etc) but haven’t found something lightweight enough , and that doesn’t require to change class hierarchies (i especially like SQlAlchemy’s approach though) (having to change the base class of an object is a big no no , because of the too tight coupling, but i ain’t 100% sure yet) but i will write more on this in another article
    • Wsgi /web server: this is one area where python definitely offers too much choice ! In the previous version, i was using Bottle a very nice “micro framework” , that is only one file ! But i am still looking around to find the right thing, and in the current version i just have a minimal , pure wsgi code without external framework , for the “WSGI interface”  part (see the different drawings ) that sort of “publishes” the different public functions of the Environment manager to make them accessible via http

    General structure changes:

    While coding and testing the different aspects of the system i realized, that, while being a nice “functionality bag” /black box to work with,  the HardwareNode structures are far from perfect:

    • it makes sense for the sensors of an Arduino to be “inside” a software abstraction of it (the ArduinoNode) : the list of sensors is actually stored inside of the the ArduinoNode data structure , and there are methods to access the sensors by type, by id , etc  .
    • but it does not make sense for a webcam for example : what would the node be ?  what would the sensor be ? In this cas there is actually not really any difference between the node and sensor, and its just confusing

    So i might alter the general structure somewhat :

    • Nodes would become Connectors: it makes sense to think of the Arduino itself, or a webcam’s usb interface as a means of communication between the sensor and the host : it is much more generic and understandable
    • Connectors would not Contain Sensors anymore, connectors would simply set themselves up/connect to the hardware , to Provide an access for sensors, would would just have a reference to the connector they use.
    • This would also reduce the effect of having to use the Node as proxy aswell as a connector : currently , to communicate with a specific sensor , you have to go through : the environment->the node manager->the node ….

    Ye old version:

    As a reminder, and since i never posted it here :this is what the project’s web based user interface looked like before the “reboot”:

    Share and Enjoy:
    • Identi.ca
    • Twitter
    • Facebook
    • Reddit
    • Digg
    • Print
    • Google Bookmarks
    • Add to favorites

    Tagged on:                                         

    10 thoughts on “HydroDuino…reborn!

    • February 2, 2011 at 20:39
      Permalink

      ie for bottle rigs use one position with an inverted u shaped tube for overflow.. the top of the inverted u is your maximum fluid level you want in your bottles to prevent overflow.
      Also use a cheap bilge pump on 12 volts for the pumping and keep on for the entire soak period and turn off when you want to drain. These pumps will NOT run at 5 volts properly. Bilge pumps are preferred because we are dealing with very salty water in hydro solutions and that water is very trashy with media in most hydro setups.

      I use a sanguino in my system with a 49.95 I2C 16 channel relay board from http://www.inexglobal.com each relay can handle dc/ac up to 240V 5 amps supposedly I use for 5 amps at 12V.
      I pump 55 gallons in about 5 minutes from a 500gph pump at 12 v. enough to fill a 4×8′ tray 8 ” deep… once at the beginning of the light cycle…

      Reply
      • February 3, 2011 at 15:28
        Permalink

        Hello Gwen!
        Thanks for the valuable info !
        *yes, bilge pumps are ideal , however usually harder /more expensive to come by, one of my sub-projects is making bilge pumps on the cheap with a reprap 3d printer
        *quite impressive setup you have, and great use of a sanguino! i must confess i work on a lot cheaper , smaller scale system 🙂
        *i had never heard of bitlash before ! Very nifty ! funny thing is , after reading through their examples, it is quite similar to my own system

        Reply
    • February 2, 2011 at 20:41
      Permalink

      the other thing I have done is to use a modified version of http://www.bitlash.net to drive the whole system with macros…

      Reply
    • February 21, 2011 at 18:47
      Permalink

      Great Work Mark!
      I have seen your entries about arduino + hydroponics in the past but Im really happy that you have renewed your interest in the project. Ive been working on basic atmospheric control and automation for my own arduino based hydro system but mine is MUCH less polished than your own. The use Python is very promising and very much over my head SO props to you for the time and energy invested in this project. Myself and many others with less polishing skills are anxiously awaiting your updates!
      Keep up the good work mate!

      Reply
      • March 1, 2011 at 01:05
        Permalink

        Hi T-Bone !
        Thanks a lot for the kind words !
        Have you been working on your project for long ? Simpler is often better ! Do you have a webpage for showcase?
        Frankly , the scope of the “new and improved” projects makes me feel like i am over my head too, sometimes 🙂
        Stay tuned for more news, and hopefully once i really feel it is right, the full software and code!
        Cheers!

        Reply
    • May 25, 2011 at 18:42
      Permalink

      The whole area of arduino / 3d printing and open source hardware is really appealing to me right now. When I thought about growing my own food i immediately thought of a hydroponics system built on arduino. I’m not a programmer though. Really interested in this project but havn’t seen any updates on Google Code page since last year. Hopefully you are still working on it!

      Reply
      • June 17, 2011 at 01:01
        Permalink

        Yes, the possibilities offered by open source hardware are really nearly limitless, and so much fun ! (althought , pretty time consuming as well 🙂

        Hydroduino was concieved from the start to appeal to non programmers as well , so there is a lot of effort beeing put into that aspect as well. I actually don’t use the google code page anymore, and didn’t want to post more articles on it because it really might seem like “vaporware” by now, however , i am still working on it, pretty much along the lines outlined in this article .

        The code will be hosted /released on github once i deem it “acceptable” (you know the saying, “when its done” 🙂
        at this adress: https://github.com/kaosat-dev/HydroDuino

        thanks for the interest !
        Are you working on similar projects (arduino/gardenning/hyrdoponics) aswell?

        Reply
    • May 3, 2012 at 12:12
      Permalink

      hey mate – are you still working on this?

      Reply
    • May 18, 2012 at 21:58
      Permalink

      Hi Mark,

      I’m a software developpment engineer , and hydroponic gardener at night….
      I wanted a DIY controller ,bc all the good proprietary one is very expensive.
      I’m starting my own controller on an arduino platform.

      Once done i will post back all schematic and software , this (i hope) will be my contribution to the DIY Grow world.

      Would you mind posting your code please, i will post you back any enhancement done .
      Would like to contribute but starting from scratch.

      Hope you will read this .

      I’ve tracked googlecode and github but none of your source file could be found.

      How to obtain your source code ? please

      Thank you

      Reply
    • March 26, 2013 at 13:31
      Permalink

      Hi Mark,

      How does one get in contact? I can’t find an email button!

      Cheers,

      Reply

    Leave a Reply to Stew Cancel reply

    Your email address will not be published.