Android Adventures part 7.5 : More multi-touch in Processing
Back to part 7…, on to Part 8…
Example sketch made with this code.
Based on my previous post (part 7), I wanted to advance the capabilities of multitouch in Processing a bit more. The previous example was fairly primitive, but got the job done. However, it had a few shortcomings:
- The drawing of graphics was happening in the surfaceTouchEvent() function, rather than Processing draw() function. This caused a bit of flickering on screen, and made it hard to screengrab stuff from the Davlik Debug Monitor.
- It only tracked the current position; it stored no historical data. What if you wanted to track the previous frames position for a certain touch point to generate a motion vector?
With a bit of work I came up with the below code. It is a snapshot of only the code that is needed to enable more robust multitouch querying in your Processing sketch. You’d take over in the draw() function. Quick overview:
- At the top two global variables are defined that store the max number of touch points your phone supports (by trial and error via this code I found that my phone has five), and an array that will store the MultiTouch data. It should be noted that you can set maxTouchEvents to a higher number just to be safe, shouldn’t hurt anything.
- setup() populates the MultiTouch array with new MultiTouch objects. Each object tracks a different touch point.
- draw() then checks for updated (touched) MultiTouch data, and then exposes those objects to you to query. The attributes I gave them mirror how Processing deals with querying mouse positional data: motionX, pmotionX, motionY, pmotionY, size, psize, id.
- surfaceTouchEvent() is a function that can be overridden to intercept Androids MotionEvent objects, that store the MultiTouch data in them. That’s what happens below: It updates our MultiTouch array created earlier with any new MultiTouch data whenever the screen has motion applied to it.
- MultiTouch is the class that stores the data extracted from the Android MotionEvent object. One is created per touch point on screen (so if you have five touch points, then you have five of these objects).
Here’s the code:
// multiTouchCore.pde
// Eric Pavey - 2011-01-02
// Code used to get multitouch working in Processing.
// Add to what's in the draw() function to implement the magic.
// It's all about the MotionEvent:
// http://developer.android.com/reference/android/view/MotionEvent.html
//------------------------------
// Setup globals:
// Define max touch events. My phone (seems to) support 5.
int maxTouchEvents = 5;
// This array will hold all of our queryable MultiTouch data:
MultiTouch[] mt;
//------------------------------
void setup() {
// Populate our MultiTouch array that will track all of our touch-points:
mt = new MultiTouch[maxTouchEvents];
for(int i=0; i < maxTouchEvents; i++) {
mt[i] = new MultiTouch();
}
}
//------------------------------
void draw() {
// If the screen is touched, start querying for MultiTouch events:
if (mousePressed == true) {
// ...for each possible touch event...
for(int i=0; i < maxTouchEvents; i++) {
// If that event been touched...
if(mt[i].touched == true) {
// DO SOMETHING WITH IT HERE:
// Amazing mult-touch graphics implemeneted....
}
}
}
}
//------------------------------
// Override parent class's surfaceTouchEvent() method to enable multi-touch.
// This is what grabs the Android multitouch data, and feeds our MultiTouch
// classes. Only executes on touch change (movement across screen, or initial
// touch).
public boolean surfaceTouchEvent(MotionEvent me) {
// Find number of touch points:
int pointers = me.getPointerCount();
// Set all MultiTouch data to "not touched":
for(int i=0; i < maxTouchEvents; i++) {
mt[i].touched = false;
}
// Update MultiTouch that 'is touched':
for(int i=0; i < maxTouchEvents; i++) {
if(i < pointers) {
mt[i].update(me, i);
}
// Update MultiTouch that 'isn't touched':
else {
mt[i].update();
}
}
// If you want the variables for motionX/motionY, mouseX/mouseY etc.
// to work properly, you'll need to call super.surfaceTouchEvent().
return super.surfaceTouchEvent(me);
}
//------------------------------
// Class to store our multitouch data per touch event.
class MultiTouch {
// Public attrs that can be queried for each touch point:
float motionX, motionY;
float pmotionX, pmotionY;
float size, psize;
int id;
boolean touched = false;
// Executed when this index has been touched:
//void update(MotionEvent me, int index, int newId){
void update(MotionEvent me, int index) {
// me : The passed in MotionEvent being queried
// index : the index of the item being queried
// newId : The id of the pressed item.
// Tried querying these via' the 'historical' methods,
// but couldn't get consistent results.
pmotionX = motionX;
pmotionY = motionY;
psize = size;
motionX = me.getX(index);
motionY = me.getY(index);
size = me.getSize(index);
id = me.getPointerId(index);
touched = true;
}
// Executed if this index hasn't been touched:
void update() {
pmotionX = motionX;
pmotionY = motionY;
psize = size;
touched = false;
}
}
Back to part 7…, on to Part 8…
Eric,
Thanks for these articles man. I dont know why there aren’t hundreds of comments here as your site is referred to in alot of forum posts and links all over the joint.
Anyways, just wanted to drop some gratitude on you. All of these articles (particularly 7 and 7.5) are life savers for everyone developing with Processing for Android. Keep up the awesome work man! Much love!
Aaron
Thanks, I appreciate it! I’m just riding on the shoulders of the giants that came before me, but it’s fun to share the view 😉
Is anyone using the “size” attribute ? I’d like to know which Android devices can report this information and with how much resolution.
Well-explained and easy to understand tutorial! I’ve got an LG Optimus One, which is a medium-end phone, and I’ve run your example on it, as well as a few examples I developed myself.
A noticeable trend in these sketches is that the FRAMERATE of the sketch dramatically drops when touch input is sensed. e.g. I get ~55FPS on your example but it drops to ~32FPS when I place even one finger on the screen.
The framerate doesn’t drop any lower with multiple fingers, though.
Just wondering if you might understand why this occurs and if you have a solution.
Thanks
Nikhil
Interesting observation, I’d not noticed that, but I’d not tried to do any profiling yet either. I don’t have any concrete ideas off hand why that would be, but plenty of speculation: Maybe my code would be written better, maybe it takes a hit when the API is accessed, maybe it’s a hardware issue on select phones, maybe it’s a Processing specific thing… only testing would tell. Good luck
Maybe it’s an issue with my phone, but I did test it with a HTC Magic I borrowed from a friend. Again, the framerates dropped from ~30 to ~15 fps. So, either it has a problem with ARMv6 devices or MDPI resolution screens. I’m pretty sure it’s not a coding problem. Have you tried displaying framerates on your device with frameRate()?
I’m hoping there’s a way to get around this. It’s probably the way certain devices handle touch events that causes the problem. Can’t find any resource about it, so hopefully you can help!
Thanks for the awesome tutorials! I have a question though…Is there any way to detect when a touch has been removed? I noticed that the surfaceTouchEvent is only called when a new touch occurs or a position changes. For example, if you have two fingers down, and then lift one, it still thinks there are two down until the first finger moves. Usually your first finger shifts slightly enough to cause another call, but if you keep your finger still enough it won’t trigger. Any thoughts? Thanks a bunch.
Good catch, I’d never realized that! Just re-ran the code and got the same results. If I ever get around to digging back into the code I’ll try and look into it.
@Aaron could’t agree more, this website is like the net’s best kept secret when it comes to Processing and Android.
Appreciate it, thanks
Here’s a solution!
http://forum.processing.org/topic/multitouch-solution-long-code
@Metabog – Just what I’ve been looking for. Many thanks.
I debug it on my Moto phone .But it only works with 2 poits.
So I change the code above.My code is below. It is very simple.
import android.view.MotionEvent;
float x1,y1,x2,y2,press1,press2;
int pointNum;
void setup() {
size(screenWidth, screenHeight);
stroke(0);
}
void draw() {
background(240);
fill(press1*200,0,0);
ellipse(x1,y1,press1*300,press1*300);
fill(0,press2*200,0);
ellipse(x2,y2,press2*300,press2*300);
}
boolean surfaceTouchEvent(MotionEvent event) {
pointNum=event.getPointerCount();
switch (event.getAction() & MotionEvent.ACTION_MASK) {
case MotionEvent.ACTION_POINTER_DOWN:
// User is pressing down another finger.
break;
case MotionEvent.ACTION_POINTER_UP:
// User is released one of the fingers.
break;
case MotionEvent.ACTION_MOVE:
x1=event.getX(0);x2=event.getX(1);
y1=event.getY(0);y2=event.getY(1);
press1=event.getSize(0);
press2=event.getSize(1);
break;
}
return super.surfaceTouchEvent(event);
}
hi,panhao,I run your sketch on my htc 328d cell phone. But it quit automatically.
My phone run in android 4.03. Do you meet this problem?
I don’t, but my phone is the same one I authored this one, which is Android 2.1.
Thanks for the amazing tutorial. It helped me a lot while trying to run Processing+Eclipse+libpd together for Android. One question though, my app has a preference screen and I am building a synth with some visuals. After I select some of the preferences from the preferenceactivity class and come back to the main screen, processing stops drawing on the screen. i tried the redraw() function but it didn’t help. Can anybody help? Much appreciated.
For some reason, when there is 1 touch on the screen it will not release. It just sticks where it last was. Is there a way around this?
hello…can you provide the full source code of this article…because when I tried to copy the codes…it gives me errors…
thank you
@Patrick
Import the MotionEvent class on the top of the script.
import android.view.MotionEvent;