Android Adventures, part 3: hardware detection in Processing
The main reason I got my Android phone was to get Processing running on it, and start grabbing the sensor data. There were no complete examples (I could find) on how to actually do this though. After two days of research I got it working. Some takeaways:
- I learned that Processing is ran as an Android “Activity“: It enters the Activity at the onResume() state, and exits it at the onPause() state. You need to override these in your sketch to do what you want. See the above link to a nice image that shows the state tree.
- All the code you need to author to setup your SensorManagers, SensorEventListeners, and Sensors, needs to happen in the onResume() function: Putting this stuff in the sketch’s setup() function won’t work.
- You can make a single SensorManager, but for each sensor you want to track you need to make a unique SensorEventListener, and Sensor.
- Once I figured it out I realized how easy it actually is. Like most things 😉
Here is a list of resources I pulled from, in order of discovery/usefulness:
- http://wiki.processing.org/w/Android – The main ‘Processing Android’ wiki page.
- http://forum.processing.org/topic/reading-the-compass – Forum reply by Ben Fry that got me started.
- http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/graphics/Compass.html – Android API example. Outdated (uses deprecated code), but good concepts.
- http://developer.android.com/reference/packages.html – the Android API docs.
- http://www.netmite.com/android/mydroid/cupcake/development/samples/Compass/src/com/example/android/compass/CompassActivity.java – Another example I pulled from.
A (cropped) screenshot off the Phone of the exciting final product!
And the code:
// android_sensorData // Eric Pavey - 2010-10-10 // http://www.akeric.com // // Query the phone's accelerometer and magnetic field data, display on screen. // Made with Android 2.1, Processing 1.2 //----------------------------------------------------------------------------------------- // Imports required for sensor usage: import android.content.Context; import android.hardware.Sensor; import android.hardware.SensorEvent; import android.hardware.SensorManager; import android.hardware.SensorEventListener; //----------------------------------------------------------------------------------------- // Screen Data: float sw, sh; // Font Data: String[] fontList; PFont androidFont; // Setup variables for the SensorManager, the SensorEventListeners, // the Sensors, and the arrays to hold the resultant sensor values: SensorManager mSensorManager; MySensorEventListener accSensorEventListener; MySensorEventListener magSensorEventListener; Sensor acc_sensor; float[] acc_values; Sensor mag_sensor; float[] mag_values; //----------------------------------------------------------------------------------------- void setup() { size(screenWidth, screenHeight, A2D); sw = screenWidth; sh = screenHeight; // Set this so the sketch won't reset as the phone is rotated: orientation(PORTRAIT); // Setup Fonts: fontList = PFont.list(); androidFont = createFont(fontList[0], 16, true); textFont(androidFont); } //----------------------------------------------------------------------------------------- void draw() { fill(0); rect(0,0,sw,sh); fill(255); if (acc_values != null) { text(("Accelerometer: " + acc_values[0] + " " + acc_values[1] + " " + acc_values[2]), 8, 20); } else { text("Accelerometer: null", 8, 20); } if(mag_values != null) { text(("Magnetic Field: " + mag_values[0] + " " + mag_values[1] + " " + mag_values[2]), 8, 40); } else { text("Magnetic Field: null", 8, 40); } } //----------------------------------------------------------------------------------------- // Override the parent (super) Activity class: // States onCreate(), onStart(), and onStop() aren't called by the sketch. Processing is entered at // the 'onResume()' state, and exits at the 'onPause()' state, so just override them: void onResume() { super.onResume(); println("RESUMED! (Sketch Entered...)"); // Build our SensorManager: mSensorManager = (SensorManager)getSystemService(Context.SENSOR_SERVICE); // Build a SensorEventListener for each type of sensor: magSensorEventListener = new MySensorEventListener(); accSensorEventListener = new MySensorEventListener(); // Get each of our Sensors: acc_sensor = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER); mag_sensor = mSensorManager.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD); // Register the SensorEventListeners with their Sensor, and their SensorManager: mSensorManager.registerListener(accSensorEventListener, acc_sensor, SensorManager.SENSOR_DELAY_GAME); mSensorManager.registerListener(magSensorEventListener, mag_sensor, SensorManager.SENSOR_DELAY_GAME); } void onPause() { // Unregister all of our SensorEventListeners upon exit: mSensorManager.unregisterListener(accSensorEventListener); mSensorManager.unregisterListener(magSensorEventListener); println("PAUSED! (Sketch Exited...)"); super.onPause(); } //----------------------------------------------------------------------------------------- // Setup our SensorEventListener class MySensorEventListener implements SensorEventListener { void onSensorChanged(SensorEvent event) { int eventType = event.sensor.getType(); if(eventType == Sensor.TYPE_ACCELEROMETER) { acc_values = event.values; } else if(eventType == Sensor.TYPE_MAGNETIC_FIELD) { mag_values = event.values; } } void onAccuracyChanged(Sensor sensor, int accuracy) { // do nuthin'... } }
Great, referenced, example…..very useful
thanks Eric, very useful!
THANKS A LOT! I could finally setup processing on my Android up to you.
just had to add a “redraw();” at the end of the onResume function cause the phone wasn’t showing anything after a pause.
Hey, that redraw() in onResume() fixed a bug I’ve been tracking for a while. Thanks for the tip!
I am sorry Mr. Eric, this is out of topic. But why I cannot find how to send and receive message (SMS) function in Processing for Android? This is very important! I need it. Thanks for your help.
Processing itself will have no libraries for that. You’d need to do it through the Android api itself. As a starting point:
http://developer.android.com/reference/android/telephony/SmsManager.html
So how to integrate automatic SMS sending with other sensor detection within Processing? Can you give some simple examples? Thank you very much for your help.
I hope that you understand what I mean. With many sensor detection capabilities, Android could become remote sensor gadget and send warning message automatically from remote area. But I don’t understand programming other than Processing itself!
Hi,
Nice post Eric, but i have a question, how to exploit magnet value, what ‘re the three value ?
I looked on the doc, on processing forum, but didn’t find explanation…
Regards Gilles
I was just grabbing them and displaying them as an example. I never put much thought into what they actually mean relative to the phone’s orientation. As far as where the three values come from, I’m not sure what you mean? You can see from the above code sample they are extracted from the SensorEvent that is passed to MySensorEventListener()
If you want to understand what those values mean relative to the phone, maybe these docs here will help you out:
http://developer.android.com/reference/android/hardware/SensorEvent.html
Has a pic showing axis orientation relative to phone, and if you scroll down talks about grabbing data from the different sensor types.
Yeah i need to look your code with more details and your fonction MySensorEventListener()
Thanks for reply
You are a hero of processing for android. I got some really cool stuff happening on my Galaxy S really quickly by using your example…thanks
Awesome, glad to help
Great stuff you’ve got here! Keep up the good work
Is it possible to use the magnetic field sensor data with a camera to achieve something like a augmented reality effect? I’ve played around with your code and camera, but can’t seem to figure this thing out. It keeps messing with the camera angels. Any ideas?
void draw() {
camera(0, 0, 0, mag_values[0], mag_values[1], 0, 0.0, 1.0, 0.0);
background(196, 236, 255);
lights();
beginShape();
vertex(-400, 400,-100);
vertex(-400, -400, -100);
vertex(400,-400, -100);
vertex(400,400,-100);
endShape(CLOSE);
}
Thanks! Worked straight out of the box so to speak (on Samsung Galaxy S Plus)…now to do something useful with this data 😉 Great work, keep it up!
Thank you very much for posting this good content! I am looking forward to checking out more!
what are the imports for the funccion ellipse and text????
Thanks very much from Helsinki!
That certainly saved me a lot of headaches
Tried it with an HTC Nexus One, and it ran without issues
( well – except the A2D didn’t work, but worked when I wrote “P2D” instead…, for some reason… maybe I set up my environment wrong…)
/miska
Glad it worked I think they may have changed the renderer since I last made these tutorials to make it more consistent when coding.
for some reason I always get the same error message: (OSX 10.7, processing 2.0a4)
BUILD FAILED
/Users/admin/android-sdks/tools/ant/build.xml:602: The following error occurred while executing this line:
/Users/admin/android-sdks/tools/ant/build.xml:622: Compile failed; see the compiler error output for details.
Total time: 1 second
When I authored this example, it was on a previous version of Processing, on Windows (not osx). Unfortunately I won’t be much help here with your error, but maybe someone else will have experienced it and provide a solution.
This is offtopic, but what about the microphone? I can’t find anything in the developer land for it.
I found out how to run this on processing 2.06b:
rename
MySensorEventListener accSensorEventListener;
MySensorEventListener magSensorEventListener;
to
SensorEventListener accSensorEventListener;
SensorEventListener magSensorEventListener;
and remove A2D as renderer so something like size(screenWidth, screenHeight);
This is a great intro to AP and sensors. I have a problem importing the Android.hardware.USB package. I get the symbol not found etc error.
I have the FTDI D311 ev kit which works fine. It acts as a host and exspects the ANDROID to act as an acessory device.. The FTDI examples GPIOdemo…work fine…BUT
I think it would be great to use processing to develop home controls . How do i get processing to see the Android.hardware.USB package..Android.hardware.Sensor loads OK.
Thanks