Netduino, HTTP, REST Web Services, and Cosm

When I first got my Netduino, I was starting with a good understanding of programming, little experience with electronics, and only a slight grasp of HTTP requests/responses. Over the past months, I've been rabidly studying all the above and I've come to understand that all three areas combine with each other and reinforce one another. They form the pillars of knowledge that one needs to develop great embedded applications for the future.

HTTP Adds a New Dimension

While all three areas are important, one of these pillars is solely responsible for a new dimension in embedded electronics: HTTP. This standard for exchanging information across distances is the backbone of the Internet and its extension to microcontrollers is what defines a movement called "The Internet of Things".  Essentially, this phrase means that devices you build can now speak and hear HTTP and therefore communicate with the Internet at large.

The book I have been reading to get started with the Internet of things is, of course, Getting Started with the Internet of Things. It is a worthy introduction to Server/Client pattern and the implementation details. Yet, I must admit that it jumps into things a little too quickly. I feel like contributing some explanations about concepts that the book doesn't cover. In particular, the book jumps into using Pachube, a web service, without fully shining a light on what it does and what web services do, in general. (Additionally, when I started with this book, Pachube no longer existed under that name. To this day, it exists as Cosm. This change made all code examples out of date, though still applicable.)

Web Services

So first off, a definition is needed.

A web service is a resource on the Internet whose main purpose is to give you data that you ask for and/or record data that you send it. 

Think of it like a function that you can call in your browser. You send it parameters and you are returned a result. The parameters might be ones that determine the data you get or they might be the data you want the web service to save (or might be the data that creates another thing that is ultimately saved). The way the web service (function) works is specific and is explained in a manual or tutorial on its website (more about this in the next section on REST).

The data being transferred in or out is usually structured in an organized way. Some formats for organizing data are commonly used in web services: XML and JSON. These formats are basically very specific ways to label data so that you know what the data means. More importantly, they are formats that a computer programming language can look at and work with for further processing.

Web services give your microcontroller four super-powers.

1. The power of information storage
2. The power to process information
3. The power to publish and share information
4. The power of memory. 

What I mean to say by this last point is that Netduino has access to the memory of the Internet. Think of this as an analogy to how you use your brain to remember past events to determine your behavior. If one day you walk through an alley lined with rancid, smelly dumpsters, you might find yourself avoiding that alley on another day because your memory of the past event is unpleasant. This is the human scope of memory, but Netduino can use any past information that exists on the internet to determine its behavior. You have this resource at your fingertips even in your smallest project.

With web services, the microcontroller can focus on the following core operations:

• Handling the event loop of your physical application
• Changing hardware states
• Processing local information

RE(presentational)S(tate)T(ransfer)

Though I'm no expert on REST, I understand it enough to be able to use web services that say they are REST-ful. As I understand it, REST is a way to structure URL for the benefit of data transfer with HTTP. In particular, each URL behaves in a particular way depending on the HTTP method that is used on the URL i.e. GET, POST, PUT, DELETE. These methods act on whatever data is specified by the URL and not all of these methods need to be supported by the web service. For example, one might not be able to DELETE all the records on a website.

When it comes to how this applies to microcontrollers, the aforementioned Cosm is a REST web service that one can send measurements to with HTTP. The website offers a nice interface to see the measurements over time on a graph (a time series).  It has an API which in other words is the manual for how to use the service to store and retrieve data.

A side note that is relevant to many web services is that once you create an account on the service, you are given a key which is a very cryptic collection of characters that works just like a key to your door. You send the key to the webservice with the HTTP request and it grants you access to the information/URLs you want.

What it all really means

Despite all the terminology, the main purpose of all these technologies is to take measurements of the physical world and store it on the internet  where you can access it anywhere you have a browser. The other main idea is to be able to react to past information and cause physical reactions in your embedded applications.

I hope this clarifies some basic concepts and that it encourages you to find better writers than me who have written about the subject and prepares you for further study.

Serving Websites on Local Network with IIS Express

Whenever I look through documentation, a tutorial, or blog post on any subject, I feel like Brigadier General Jack D. Ripper in Dr. Strangelove, except it's "Clarity of Essence" I'm after and not "Purity". I go to great lengths to find resources that speak to me, but sometimes when a problem needs troubleshooting it's impossible to find anything but poorly worded (and typeset) posts. But, rather than complain, I think I can contribute to some documentation on a problem whose tail I was chasing for too long.

The modern goal of embedded electronics is to create a world of "smart" devices that can upload information to and be controlled from the Cloud -- also known as the Internet to the meteorologically disinclined. Such a networked group of devices makes it possible to monitor and control our stuff from mobile devices to the benefit of our convenience and the reduction of our energy consumption. From an extreme perspective, this turns our world into a sort of video game where we can lazily monitor our current states, stats and change things with the press of a button or tilt of a joystick. I don't necessarily advocate achieving that sort of control. Certain applications of this level of control rattle my conscience (drone warfare). However, it goes without saying some applications need to be around for the good of the environment and humanity. 

Because I'm interested in building this type of networked devices, I consequently have been learning to be able to build embedded circuits and create website interfaces. My tools of choice, the two most elegant tools for doing this, are Netduino and ASP MVC. Netduino allows you to program circuits in C# and provides access to the elegant .NET Micro library. The ASP MVC framework is a great tool for quickly building websites that will offer a REST API, be backed by a database, and will benefit from templated webpages. It too relies on C#, so all logic on both sides is written in the same language.

After building a simple website with ASP MVC, I wanted to be able to send requests to it on my local network from Netduino.  To see if this was possible, I needed to first try to connect to the website from my Android phone. When I ran my website from Visual Studio, it only served the website to "localhost" on a particular port (http://localhost:<5 digit port>/). I couldn't type this URL into my phone and access the site.

I found out that Visual Studio runs the project and serves it by using IIS Express, a scaled down version of the commercial IIS server developed by Microsoft. In the project properties, there is a field to specify which URL you want to address the site. By default it is what I already mentioned, localhost. However, there is a way to make IIS Express serve the website when it receives a request for a different URL. Once this is achieved, changing what's called a URL reservation and security settings allow it to serve to computers, or phones, on the local network.

Here are the steps to follow:

1. Find Documents/IISExpress/config/applicationhost.xml. This has a section <sites> and in it there is a section <bindings>. These list all the URLs that will represent the site. Copy the existing localhost binding and replace localhost with your IP address. Find your IP address with ipconfig in the command line and look at the top of the output for the IPv4 label.
2. Add a URL reservation. This will grant the right for others on the network to request a URL from IIS Express and is done at the command line by typing:
• netsh http add urlacl url=http://<your IP>:<5 digit port>/ user=everyone (Italics represents parameters to be passed.)
3. Add a firewall rule to allow inbound requests on the port. Go to Control Panel > Windows Firewall > Advanced Settings > Inbound Rules > Add Rule. Name the rule after your site (since it will be the only one running on the chosen port). Change the scope Remote IP Address to "Local subnet". Make sure Profiles is limited to Private. Protocols and Ports should be set to TCP and the port you found in Visual Studio. 

I found that I didn't need to change the URL specified in the properties of the project in Visual Studio so long as it was localhost. In fact, it even worked when I changed it to the IP address URL. 

The problem with all other walk through posts is that they recommended the use of machine name. This never worked for me. The way I described does not require the application to be run with administrator privileges or other nonsense. 

In conclusion, I haven't created any applications that connect the Netduino and my ASP MVC project yet, but at the very least, I know I can access my service on my local network since my phone can browse the site. More to come.

A Micropost on Microcontrollers

Microcontrollers allow you to control circuits by using a programming language. Real, physical circuit components can become variables in your virtual programming world and you can act on them as you would on virtual objects in your program. Since programs can access networks, one can collect data produced by a circuit, process it into information, and send it to a remote database via the internet, in effect automating scientific data recording. The reverse process allows information to be sent to the microcontroller via the internet (from the result of a website interaction) and the behavior of the circuit can be changed based on those remote inputs.

The programs that control your circuit would live on the microcontroller and they are able to read voltages and produce voltages in the circuit. When voltages are read, they represent different states of the circuit e.g. whether a button is on or off or a sensor measurement. When voltage is produced, the program can alter the behavior of the circuit. When reading and writing are combined, one can utilize feedback and alter the behavior of the circuit based on the states it is in. The produced voltages can provide enough power for smaller circuits, but if more power is needed, smaller circuits can be activated with this lower voltage to drive higher power applications.

Microcontrollers are a tool that allow for, oftentimes, a simpler, more efficient, and faster implementation of various circuit behaviors. This is not to say that they are more energy efficient than a well designed, single-purpose circuit. For example, if we need to build a circuit with an LED that blinks every second, this can be implemented without a microcontroller, however by using a microcontroller, we have the ability to easily code this behavior using object-oriented methods. If we want to tailor the light's behavior later to change its blink rate, for example, this can be done with lines of code instead of wires and capacitors. The power and simplicity of a programming language allows us to manipulate the electrical inputs and outputs without knowing about different circuits and circuit components that would create the behavior we want. This allows a single circuit connected to a microcontroller to have an incredible number of behaviors.

The microcontroller is connected to a program virtually, however to be useful it must be physically connected to the world with sensor circuits and actuator circuits. Sensors take the world and convert it to electricity. Actuators take electricity and change the world. The microcontroller is the bridge between ideas and physical form.