Automating Android Builds: Part I


The Android platform provides a great way to deliver rich and highly functional native mobile applications to an ever growing number of smartphone and tablet devices, and soon even your TV. With this platform comes a rich set of development tools, primarily based around the Eclipse development environment, but which also includes command line tools for packaging, installing, and debugging Android applications, as well as helper tasks for Apache Ant which is used for build automation for many enterprise Java projects.

Most development teams working on complex projects in this day and age are going to employ some combination of generally accepted best practices in software engineering to ensure consistency in build quality, automate complex build processes, and automate unit and functional testing to continually ensure quality as development progresses.  Such practices include:

  • Automated unit, integration, and functional testing (faciliated by tools such as JUnit and Selenium)
  • Build automation (using tools like Ant and Maven)
  • Continuous build integration (to ensure build quality and identify conflicts early)
  • Code coverage analysis (measuring how much of your application you’ve tested)
  • Static code analysis (find common coding errors with tools such as FindBugs)

It is reasonable to assume that a complex Android application would benefit from these practices, especially with a larger development team.

Android provides most of the necessary integration points and tools to implement many of the best practices mentioned above, and allows for automating the several steps required to build an Android application.  However, despite this, there are some key lessons to learn that to learn that separate an Android build process from other Java-based projects.

Java is not Java

First off, let’s get one thing straight.  The Java classes that one writes to implement an Android application compile to the Dalvik VM, not the Java Virtual Machine provided by Sun (now Oracle).  The engineers building the Android platform chose a separate virtual machine implementation for a variety of reasons, but it is arguably the need to slim down and tailor a VM for use on a mobile device, which has very hard limitations on processor resources and memory, that lead to this decision.

Because of the separate VM implementation, the full set of standard Java SE classes is not available.  The Android team has provided a workable subset of the Java SE API with a few notable additions to assist in such things as managing HTTP communication, parsing JSON, and parsing XML.  This distinction leads to one very important point:

Not all Java targeted for the Sun JVM will run in Android.

This is important to note when planning to use 3rd party libraries in an Android application that were not targeted for Android.  While many 3rd party JARs can be included in Android projects, many cannot.  Clearly if the dependencies can be met with the pared down API, the JAR will not work.

Running It

The APIs provided by Android’s augmented subset of the Java SE API are exposed to developer projects (primarily in Eclipse) via the android.jar file.  This JAR provides the interfaces for the Android API, but none of the implementation code.

Why the limitation?

Classes compiled for the Dalvik VM aren’t executed in their .class file bytecode form, but rather must be converted to Dalvik’s Executable format, or dex.  According to this Android page, dex is optimized for a smaller memory footprint, and also to allow multiple VMs to execute efficiently.

What this means for the average developer is that you cannot go to your trusty Eclipse Run As > JUnit Test menu item and fire off unit tests.  All Java code targeted for Android must run within the Android execution environment, compiled as .dex files, in the Dalvik VM.  Without the implementation classes for android.jar being in standard JAR format (they’re buried in .dex files in Android), the best options for running your code during development is either:

  • Plugging in your favorite Android phone via USB debugging or
  • Running your code in an Android emulator

While there are ways to run the Dalvik VM on the desktop, those methods are not yet integrated into the Android standard toolchain.

Emulating Success

The Android SDK provides a rich set of tools for creating, launching, customizing, and debugging emulated Android devices, known as Android Virtual Devices (AVDs).  These emulator instances can model different hardware configurations as well as different target Android OS versions, which is useful for testing an application across the many deployed versions of Android that exist on the many Android devices that exist today.

The Android emulator, based on QEMU, provides emulated versions of many of the hardware features commonly found on Android devices (from the Android Developer Center):

  • An ARMv5 CPU and the corresponding memory-management unit (MMU)
  • A 16-bit LCD display
  • One or more keyboards (a Qwerty-based keyboard and associated Dpad/Phone buttons)
  • A sound chip with output and input capabilities
  • Flash memory partitions (emulated through disk image files on the development machine)
  • A GSM modem, including a simulated SIM Card
While this represents a common set of hardware features, it does not provide the following (from here):
  • No support for placing or receiving actual phone calls. You can simulate phone calls (placed and received) through the emulator console, however.
  • No support for USB connections
  • No support for camera/video capture (input).
  • No support for device-attached headphones
  • No support for determining connected state
  • No support for determining battery charge level and AC charging state
  • No support for determining SD card insert/eject
  • No support for Bluetooth

Beyond these official limitations, I’ve found a few more.  The emulator is often quite slow.  I haven’t spent the time to quantify this but it is slow in a few key areas.

The time to startup a “fresh” emulator is often too long for casual unit tests, upwards of several minutes, which is why the Android Eclipse tools allows the user to keep an emulator running between test runs.  This will have impact on our testing automation in the upcoming Part II of this article.

The slowness is evident elsewhere.  It is a generally held belief that the emulator does not adequately support video playback, and my experience in building custom video players in Android supports this.  Playback often does not come close to the video frame rate, therefore video playback testing is best handled on a physical device via USB debugging.  Given that it is difficult to automate testing around video playback, this is less of a concern, however it is important to note when setting up a QA team for playback testing.  Real devices should be available for QA as the emulator will likely not suffice.

Up Next

This is just a taste of some of the relevant architectural details of the Android platform that affect build automation.  In the next installment, I’ll dig deeper into build configuration, automating tests and managing the emulator from Ant.

The World Cup of Online Video – what does it mean?


Well, the stats are in, and it’s official. The World Cup – considered by many to be the largest, most watched sporting event in the world – did in fact post the largest online video viewing numbers of all time as measured by a whole host of outlets and measurement services. Big deal, right? It’s a no brainer that we’d see such staggering numbers for the World Cup – I mean, it’s the bloody World Cup! One of a handful of events in our globalized, 24/7 media input world that can still rally that kind of focus and participation. So – yea – it’s no surprise we broke records for online viewership. Why should we care anyway? And what does this mean about how our companies and organizations should be thinking about online video now and in the immediate future?

The main reason why I cared is not because I’m a diehard soccer fan, but because we just expected to be able to watch the games. Live. Wherever we were at that moment. In good quality. With good audio (vuvuzelas notwithstanding). And a host of other features to keep us interested. And we did. Online. Over the phone. It’s amazing how quickly we’ve come to expect that we can consume a live event of the magnitude of the World Cup online. I mean – that is cool.

But does it matter to your business? How does this relate at all? In some ways, of course, it doesn’t. Sports have been a driver for many of the mega-events in recent online video history, primarily due to the nature of the events themselves. But outside of the media world – what does this mean for your online video strategy:

Go global. If your company has a global presence this event certainly confirmed what you may already know….that audiences around the globe have the capability and desire to consume online video. But don’t just throw anything out there. Make sure you provide the proper translations and content to fit the needs of the region you are targeting.

Interactivity = engagement. The majority of the implementations had a high level of interactive features woven into the experience. The ESPN3 player allowed the ability to switch between games, languages, get stats, chat with others. It’s not just about providing the moving pictures. The ability to customize players with interactive details and hot spots is growing quickly and provides many businesses with the ability to create more engagement and calls to action towards your ultimate business goal.

Be where your users are. The success of online video with an event like the World Cup hinged directly on providing fans the ability to get their fix wherever they were – at work, on the road, at home. This means thinking not just about video on your website but also how to syndicate your assets externally and how you might leverage mobile platforms to deliver your message to your users.

 

 

 

Sprinting with Rails 3


I love Ruby on Rails.  There are few things in life that bring me more joy than opening up Textmate and coding a Rails web app.   That being said, my infatuation has been with Rails 2, and I’ve been itching to get working with Rails 3 – especially with the prospect of a release candidate in the near future.    So, I spent some time playing with the new framework, and I feel like I’ve fallen in love all over again.  The goal of this post is to showcase some of the awesome new features in Rails 3 while building a generic, reusable application skeleton which follows Rails best practices.

1. Generating the app

To start things off, create a new project (I’ll call it “sprinter”) and set up the Git repository.
> rails new sprinter --database=mysql
> cd sprinter/
> rm public/index.html
> rm public/images/rails.png
> git init
> git add .
> git commit -m "Initial creation of the Sprinter app"
> mate .

As you can see, I didn’t even think about a .gitignore file.  Since the Rails community has fully embraced git as the source control system of choice, Rails 3 provides a default .gitignore file for you.

Side note: if you haven’t gotten up to speed on using git yet, you should.  GitHub has a nice crash course.

2. Configuring the app

The first place to start is the Gemfile.  This is the new place to declare all your gem dependencies (it used to be in “config/environment.rb”, which, by the way,  is now “config/application.rb”).    Once you open it up, you’ll see it’s pretty much the same as how you declared gems in environment.rb, except you wont be typing “config” over and over.

Here’s the Gemfile I use for this project:

source 'http://rubygems.org'
gem 'rails', '3.0.0.beta4'
gem 'mysql'
gem 'mongrel'
group :test do
  gem "shoulda"
end

Normally, I would include a few more gems here (specifically: HAML, friendly_id, authlogic, and paperclip), but this is just a quick starter app, so I omitted them for now.  I included “shoulda” in the test group simply to illustrate how to include gems purely for a test environment.  Plus, you should be using shoulda.

3. Bundler

Now that the gems are defined, I can use Rails 3’s new bundler to make sure all required gems are installed.  This is how you can guarantee every system is using the right gems for the project.  It’s similar to requiring gems in environment.rb, but bundler allows you to package the gems in the deliverable.  Also, bundler is far smarter than the previous system.  Previously, gem dependencies were read linearly out of config/environment.rb which had the propensity to cause issues.

For example, consider the following scenario.

  • You have versions 1.2.3 and 1.2.4 of a gem (let’s call it “some-gem”) installed on your system.
  • Your app depends on “some-gem” version “>= 1.2.3″, which is declared in config/environment.rb
  • A different gem required by your app depends upon “some-gem” version “= 1.2.3″

Since you have version “1.2.4″ installed, the “>= 1.2.3″ requirement would use the version “1.2.4″ installed on your system (because it is the latest, greatest version).  However, when it later reads that version “= 1.2.3″ is required, it fails since it already loaded version “1.2.4″.   It’s an edge case, but Bundler will prevent this from happening.

Anyway, I got a bit off subject there.  The next step is to use bundler to make sure all those gems are installed.

> sudo bundle install

You’ll see it fetch the correct versions of all the gems defined in the Gemfile.

4. Using jQuery

One of the best features of Rails 3 is that all the javascript is unobtrusive, thus being fairly framework-agnostic.  Since I’m a big fan of jQuery, and less so of Prototype, let’s swap in jQuery by doing the following:

  1. Delete all .js files (except for “application.js”) in the “public/javascripts” directory.
  2. Grab the “rails.js” file from the src directory of http://github.com/rails/jquery-ujs and place this in the “public/javascripts” directory.

Believe it or not, you now have everything you need to use jQuery.  To make things more seamless though, you should create file “config/initializers/jquery.rb”, and paste the following inside:

module ActionView::Helpers::AssetTagHelper

  remove_const :JAVASCRIPT_DEFAULT_SOURCES

  JAVASCRIPT_DEFAULT_SOURCES = %w(

    http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js 

    rails.js

  )

  reset_javascript_include_default

end

This will include the Google CDN hosted jQuery (saving you some bandwidth and speeding up your pages) when you use the “:defaults” javascript include tag.

Side note: Because all the javascript is unobtrusive,  some of those fun helpers like “link_to_function” no longer exist.  However, “link_to_remote” still exists, but you call it as a regular “link_to” with “:remote => true” as an option.

It’s one thing for the framework to try and enforce unobtrusive javascript, but we should also set up our code to enforce it as well.    At this point, I usually create the following helper methods in “app/helpers/application_helper.rb”:

def stylesheet(*args)
  content_for(:stylesheets) { stylesheet_link_tag(*args) }
end
def javascript(*args)
  content_for(:javascripts) { javascript_include_tag(*args) }
end

These functions help me group all the stylesheet and javascript includes together.  With that, I then update the application layout file like so:

<!DOCTYPE html>
<html>
<head>
  <title>Sprinter</title>
  <% stylesheet :all %>
  <% javascript :defaults %>
  <%= yield(:stylesheets) %>
  <%= csrf_meta_tag %>
</head>
<body>
  <%= yield %>
  <%= yield(:javascripts) %>
</body>
</html>

Another side note:  Rails 3 spits out pure HTML5.

Moving the javascript include tags to the bottom does several things:

  1. Any inline-javascript that depends on jQuery or any of our other methods will cause an error.  This forces you to use an external file for it (like you should).
  2. There is no browser blocking when loading the javascript files, thus causing the page to load faster.

5. Rails 3 Routes

Now that all the required gems are installed, we’re ready to roll.  I like to start by creating a simple “Welcome” controller for the app.  One thing to note is that all the Rails 2 “script/” commands are now part of the “rails” command.

> rails generate controller Welcome index

Followed by an update to the “config/routes.rb” file to set the “welcome” controller as the root of the project.

root :to => "welcome#index"

In Rails 2, you would have declared the same thing with something like:

map.root :controller => 'welcome', :action => 'index'

Rails 3 routes are built to be  more extendable and require a lot less typing (notice the “controller#action” naming convention).  Let’s make sure everything’s working:

> rake db:create
> rails server

The site should now be visible at “http://localhost:3000″.  When you’re satisfied, stop the server with Ctrl+C.

6. Scaffolding

So far, I think this is a great starting point for quickly getting up and running on a new Rails 3 project.  However,it really doesn’t do anything yet.    So, let’s make it into a simple blog.

> rails generate scaffold Post title:string body:text published:boolean

This looks pretty standard, but a few key things have changed for Rails 3.  One, the scaffold generates a “_form.html.erb” partial instead of repeating the form in the “new” and “edit” views, making things much DRYer.

This also sets up the routes like so:

resources :posts

Just to close the discussion on routes, consider if posts belonged to a user and also featured a “comment” action.  We could set the routes up like so:

resources :users do
  resources :posts do
    member do
      get :comment
    end
  end
end

Pretty clean, isn’t it?

Anyway, back to the app.  Let’s see how things are working.  Run:

> rake db:migrate

Starting the server and checking out “/posts” gives you the standard Rails scaffolding page.  All is good here.

7. And finally, Arel

Now I can introduce my favorite part of Rails 3: Arel.

In a nutshell, Arel is an ORM-agnostic framework based on Relational Algebra (what the SQL language is based on).  I’m sure I’ll be blogging about it in far more detail shortly, but it essentially allows you to make all your Rails queries chainable (much like “named_scope” in Rails 2 – in fact, it was written by the same guy).

Open up app/controllers/posts_controller.rb and we’ll give it a shot.

In the “index” action, you can see that it grabs the blog posts in the standard Rails way:

@posts = Post.all

Normally, to only show the latest 25 published posts, we’d change this to be:

@posts = Post.all(:conditions => { :published => true }, :limit => 25, : order => "posts.created_at DESC")

With Arel, we can make this awesome by writing it as:

@posts = Post.where(:published => true).order("posts.created_at DESC").limit(25)

You should notice that, apart from a new naming convention, there is one major difference:  in Rails 3, we don’t call “.all”.  This is by choice.  We could call .all on that line and everything would work as expected (and in some cases, you want to call it).  However, calling “.all” forces the query to be run immediately.  But, thanks to Arel, if we leave it off, the query is only run when we actually use the data.

So, our query is only run when the view hits:

<% @posts.each do |post| %>
...
<% end %>

I don’t know about you, but I think that’s awesome.  It’s cleaner, and far more extendable.  For example, say we wanted to have a toggle switch for published and unpublished posts.  Our “index” action could be something like:

@posts = Post.order("posts.created_at DESC").limit(25)
if(params[:draft])  
  @posts = @posts.where(:published => false)
else  
  @posts = @posts.where(:published => true)
end

Gorgeous, isn’t it?

Well, that’s it for now.  I hope this post helps you hit the ground running with Rails 3.  I’ll be sure to post more about Rails 3 as I discover it, but so far, I’m extremely excited about it.  I hope you feel the same way.  Leave your thoughts/gripes in the comments.

Grails – Exporting / Importing Domain Objects using DefaultGrailsDomainClass


Lately I’ve been working on a multi-tenant web app that contains a good-size number of domain objects. Within these objects, there exists a sub-set belonging to a root object, and the need arose to be able to quickly duplicate / populate the data within these objects between different instances of the app. It has become quite an interesting problem: certainly one could create some SQL scripts to export/import data, but how does one do it in a user-friendly, Groovy/Grails manner?

The key turned out to be the discovery of the DefaultGrailsDomainClass . This class performs a series of introspections on your domain objects, and allows you to programmatically access quite a bit of information on them. For example, suppose we had the following domain object:

class Pizza{
String name

BigDecimal diameter

List toppings

hasMany = [toppings:Topping]
}

Class ‘Pizza’ contains a name / description, diameter, and a many-to-many relationship with the ‘Topping’ class (although not defined here). Nice and simple… now, the good stuff.

By passing a Domain Object class to DefaultGrailsDomainClass, we can access a good deal of info about that Domain Object. Among other things there are methods to access a Map of the constraints, determine whether or not a property is a relation (1:1,1:m,m:m), and access the properties directly by obtaining a set of GrailsDomainClassProperties . Of particular use to me is the DefaultGrailsDomainClass.getPersistantProperties() method, which returns a GrailsDomainClassProperty List of all the properties that are persisted to the database. That’s pretty excellent.

To illustrate this, let’s get the persistent properties for a Pizza:

def pizzaProperties = new DefaultGrailsDomainClass(Pizza.class).getPersistantProperties()

pizzaProperties now contains a list of ‘DefaultGrailsDomainClassProperty’ objects. If we iterate through them, we can see the following information:

[name=diameter,type=double,persistent=true, optional=false, association=false, bidirectional=false, association-type=<null>]

[name=name,type=class java.lang.String,persistent=true, optional=false, association=false, bidirectional=false, association-type=<null>]

[name=toppings,type=interface java.util.List, persistent=true, optional=true, association=true, bidirectional=false, association-type=one-to-many]

 

 

Think of the possibilities of what can be accomplished by programmatically accessing the properties on your domain object! However, I am not exceedingly clever:

def exportClassSingle(ConfigObject co, String className, objInstance){
if(objInstance){
/*
The DefaultGrailsDomainClass allows discovery of the persistent properties for a domain object.
*/
def props = new DefaultGrailsDomainClass(objInstance.class).getPersistantProperties()
/*
getPersistantProperties generates a list of “DefaultGrailsDomainClassProperty” which contains a

good deal of info about each property.

 

*/
for(p in props){
if(!p.isAssociation()){
co[className][p.name] = preExportClean(objInstance[p.name])
}
else{//do something clever!} }
}

The exportClassSingle method above accepts the ConfigObject which will be populated with the domain object’s data, the name of the class, and an instance of the domain object. It is relatively simple: it takes the class, loads its persistent properties, loops through through them and if the property is not an association, it is added to the config object.  By repeatedly calling the method for each of our Domain objects, we build up the ConfigObject which contains class and instance data. The preExportClean() method looks for and escapes any usage of ‘$’, which can cause some odd behavior if not escaped. We ignore associations here because in the real app, everything is connected to one root object; if there were associations among the child objects that we wished to capture then the above method would be quite different.

The resulting ConfigObject can then be converted into a String containing a DSL describing your objects and data. The string can then be loaded back into a ConfigObject using the ConfigSlurper.

Given the following DSL:

pizza{
pizza_0{
name="Small"
diameter="8.0"
}
pizza_1{
name="Medium"
diameter="14.5"
}
pizza_2{
name="Large"
diameter="21.0"
}
}

and assuming it’s saved in a String called ‘pizzaDSL’, we can import this data into our system using the following steps:

1. Import the DSL using ConfigSlurper.

ConfigObject config = new ConfigSlurper().parse( pizzaDSL )

The ConfigObject config now contains a map of ConfigObjects. Using the key of ‘pizza’ returns a map of 3 more ConfigObjects.

2. Construct objects using the configObject.

config["pizza"].each{key,map->
Pizza p = new Pizza()
p.properties = map
//set toppings
p.save()
}

One could also iterate through the root config object (in the case where we had more objects, e.g. ‘topping’) and use the key to determine which object to create and populate. This simple case has only one class, Pizza (and thus only one key).

3. Celebrate

Well, that’s it. I hope this helps explain the usage of the Default Domain Class and that you all find a usage for it.

Previous Articles

Googlebook?


Thoughts from Streaming Media East


Gory Details of In-App Purchases


Creation and Collaboration on the iPhone


Calling all Technical Architects and Project/Program Managers!


Google Analytics Event Tracking now available to all accounts


Welcome to Cantina Consulting

We provide experienced, senior-level technical consultants that work seamlessly with client teams to help deliver complex internet projects while meeting tight time deadlines and budget constraints.