I have done some work to help me understand multitouch features in Android. I built a simple demo with examples of pinch zooming and panning. Most of the examples involve using drawing operations on the canvas of a view.
About two years ago, I started some work on moving images in Android. I started with the “Making Sense of Multitouch” blog article and ended up doing some simple things with moving images and then drag-drop based on the Android Launcher (see references for links). It’s interesting to me that I have gone back to that same article for a different purpose. I am working on apps where the basic zoom and pan operations are needed.
Overview of the Examples
1 Basic Multitouch
The first example does the basics of multitouch as explained in the Multitouch article. An image displays on the screen. You can touch the image and move it around. You can also use two fingers to do pinch-zoom and zoom out.
I did this as the first step because I wanted to be sure that I understood the Multitouch article and had a working example.
To understand how this example works, go through the Mulitouch article and then get the source code for my examples (see below). My adaptation of the Multitouch article is in class PanZoomView.
/** * This view supports both zooming and panning. * What gets shown in the view depends on the onDraw method provided by subclasses. * The default onDraw method defined displays R.drawable.zoom_view_sample. */ public class PanZoomView extends View
The key things to understand in the article and the code are these:
- How the canvas moves around. See the call to “canvas.translate(x, y)” in method onDraw.
- How the canvas is scaled during a pinch-zoom gesture. See the call to :canvas.scale(mScaleFactor, mScaleFactor)” in method onDraw.
- How onTouchEvent handles touch and multitouch events. It sets variables so the calls to translate and scale work.
2 Canvas
In the second section, there are two examples that do essentially the same thing for handling panning and zooming as the first one. The difference is that instead of an image being moved around and zoomed, it is a circle and a rectangle. Those are drawn on the canvas in the onDraw method of the view.
I did these as the next step in understanding how to build my views for my own app. Zooming an image was step one. Zooming a custom canvas was step 2. To help me understand pinch zoom and the point around which zooming occurs, I display a small circle at the point of focus for the scaling of the view. Place two fingers on the screen and look for that dot, halfway between your two fingers.
The Circles example supports both panning and zooming. The Rectangles example supports only zooming. I found myself getting a bit confused while trying to understand for translation on the canvas and scaling so both examples were useful to me.
3. Pan Zoom Listener
PanZoomListener is an interesting example. It comes from Java Pan / Zoom listener for Android (reference 2 below). Rather than doing a custom view every time you want to have panning and zooming, which is what all the other examples do, you simply set up a listener on a view you already have working.
FrameLayout f = new FrameLayout(context); FrameLayout.LayoutParams fp = new FrameLayout.LayoutParams (...); imageView imageView = new ImageView (this); view.addView (imageView, fp); view.setOnTouchListener(new PanAndZoomListener(view, imageView, Anchor.TOPLEFT));
That may look like a lot to do, but it’s not nearly as complicated as what you do when you build a view of your own with custom handling of panning and zooming.
It works pretty well, but there are a few things that you’d need to adjust because not all of the operations are completely smooth. I definitely like the idea. I adapted the code and built my own example.
What you see in the example I did is an image. As soon as you touch it, it resizes and centers itself. After that, you can use pinch zoom on it. The focus of the zoom is the point between your two fingers.
4. Image Squares
This example was another step toward getting the right panning and zooming effects in the app that I am working on. In the app, I want to be able to draw a grid of squares on a canvas. In the example, I don’t draw the entire grid but simply some squares along a diagonal from top to bottom. This example supports zooming but not moving the canvas around as you touch the screen with one finger. The focus point of the zoom is shown with a small circle again.
You cannot move the canvas around with a simple touch. However, if you start a pinch-zoom gesture and then move both fingers around the image does shift. I never figured out exactly what was causing that. So watch out for that if build on this example.
5. Fixed Point Zoom
Since I was having a few problems with the canvas jumping around a bit, I decided to try using a fixed point for zooming. The last example zooms in and out around the center dot shown on the screen.
The code structure of this example is a bit different than the earlier ones. All of the views in these examples are subclasses of a PanZoomView class. The view used in this example is LargeGridView. It defines its own drawOnCanvas method like all the rest do, but it also defines its own onDraw method. By the time I got here, I found that the flexible framework I built in PanZoomView to explore the different aspects of panning and zooming wasn’t quite right for fixed point zooming.
Fixed point zoom is what I have ended up using in a couple of apps.
Conclusion
I have done this article a bit differently than some of my earlier blog tutorials. I have not included as much code and explanations. I assume that most readers will dig into the code as they need to. Download the source code using the links in the next section.
What I wanted to do here is show the progression of my thinking and understanding as I worked on an interesting problem. If you are new to Android, I suggest following a similar progression. Start simple and add more complex features as your understanding increases.
Source Code
You can download the source code for this demo from the wglxy.com website. Click here: download zip file from wglxy.com. The zip is attached at the bottom of that page. After you import the project into Eclipse, it’s a good idea to use the Project – Clean menu item to rebuild the project.
This demo app was compiled with Android 4.2 (API 17). It works in all API levels from API 10 on up.
Source code is also available as a zip file from Google Drive.
References
Making Sense of Multitouch – I recommend this to anyone looking for a good starting point for touch handling in Android.
Java / Pan Zoom Listener for Android - Article that describes the PanZoomListener. Definitely, take a look at this one.
Moving Views in Android – Part 1 – a simple example that moves an image when you click a button
Moving Views in Android – Part 2, Drag and Drop – an example based on the Launcher code of Android 2.2.
Neat. I just wish that there was a good way to handle large bitmaps in Android on the 2D view, without instantly running out of memory. Pinching to zoom isn’t super-relevant if the image is too low resolution to be usable.
It surprises me a bit that there don’t exist any really low complexity OpenGL frameworks/examples for Android (i.e., Canvas with OpenGL).
I am working on two apps where I have to be careful with memory. In one app, I create a canvas that is three times the shortest screen width and I am to zoom in and out on that. For example, on a Nexus 7, it is roughly 2400 x 2400. Since there is only one of those bitmaps active at a time, the app does not have memory problems. In another app, users can submit images and pictures from their phone or tablet to a server and then browse the set of all images from all users on their device. I maintain a cache of images in memory and on the SD card and size internal bitmaps so they are reasonable for the screen of each device. I learned all of this from a great article and working example on the Android Developers website: Display Bitmaps Efficiently. You might want to read that and see if it gives you any ideas on how to manage images in your own app.
The Nexus 7 has a generous 48Mb heap, doesn’t it? So not really a problem with a 24Mb image for that device – you’d have trouble running it on a 16Mb mdpi device (13Mb size). Given the number of OOMs I still see regularly, I’m pretty sure there are still some pretty low memory devices out in the wild (in my most popular app, I use a 1024×1024 base image that I overlay smaller sprites on). OOMs are – by a huge margin – the most commonly reported error.
To be fair, it has a lot of users and most people apparently don’t have any problems, but it is still extremely annoying to see those 10+ reports every week.
I use a singleton “ImageManager” class, which loads the images using SoftReferences, pretty much as accepted wisdom dictates. The only image I cannot soft-load is the main image, as the accidental purging that would kill graphical performance. Apparently, that locks up enough memory to cause the OOMs on some devices. And I’m not even convinced soft-referencing the large image would help either – Android is just slow at recovering memory from images.
One approach I’ve tried is to tile large images, and that seemingly works – albeit with some occasional stuttering from time to time if scrolling goes a bit too fast. It wouldn’t work for zooming, though, as one would still have to get the entire image into memory if the thing is completely zoomed out.
In “Loading Large Bitmaps Efficiently“, they talk about loading a scaled down version of a bitmap into memory by setting inSampleSize. Is that what you do when running on a 16Mb mdpi device, which I assume is much lower resolution than a Nexus 7? Wouldn’t that way of loading let you choose a reasonable size that would still fit in memory on the smaller device? Or is the loss in quality of the image unacceptable?
I never used inSampleSize on 16mb devices for 1024×1024 images; in theory, they shouldn’t cause problems (practice is another matter of course).
Low resolution images are not very useful games. The problem with all of
Google’s comments on handling of large images is that their default answer is always: “well, you don’t _really_ want to use that large image, do you?” But yes, yes I do.
I’ve never considered using inSamplesize for zooming. Loading in new scaled down images in the middle of updating the screen sounds like the sort of thing that would lead to even more stuttering as the old image set gets purged – and the tiled approach is already noticeable enough as is that I’m not sure I’d want to use it “in the wild”.
Haven’t tried to do it, however.. Might be worth a try, though at the moment I’m leaning more toward looking at something like this very simple OpenGL skeleton code (https://github.com/costimuraru/Simple-Android-OpenGL-2D-Rendering), and seeing if I can make time to extend it into a very simple 2D rendering framework for large image scrolling/zooming – assuming one can move most of the textures out of the heap.
Meh – I’m just grumbling because I work on a corner use case that very few others seem to care for.
The time I used this it worked out well. I started with the high resolution image, but when I was ready to display it, I used inSampleSize to reduce the size of the bitmap I created. I did some experimentation so I knew what sampling worked for various device screen sizes and densities. I tried not to sacrifice quality to the point where the images looked bad.
Well, I guess it is something I will have to look into. Actually, simply discussing this got me annoyed/interested enough to do some research on the problem again, and I fell across this OS project which seems to do exactly what I’d like to be able to do:
https://github.com/johnnylambada/WorldMap
Will need to take a look at that, and see if I can make something of it that works. It bugs me enough that I’d seriously consider taking the effort to clean up my code and open-source it if I managed a good “generic” solution.
The WorldMap example that I linked above gave me the final pieces of the ideas required to solve this problem to my satisfaction, and – having made a basic implementation that I’m satisfied with, I’ve now released it as open-source.
https://github.com/micabyte/android_game
The idea is basically Lombardo’s, but I built it using a SurfaceView/Renderer structure and – like you discuss above – chose to zoom around a center point (Lombardo for some reason chose to zoom around the left upper corner, which really doesn’t work as a generic solution).
Still needs a bit of work. The trick is really in the idea of using a downsampled version to cover up while retrieving the “real version”, but it doesn’t look as good as it could when there is a big gap in size between the detailed and downsampled version. One thing that could be useful would be let the background dynamically downsample what it fetches depending on the ScaleFactor. This would make the transition from highres to lowres less jarring when working with those really huge images. I should probably also do a few examples to demonstrate how this can be used.
Thanks for the discussion and all the stuff you put on your blog.
Thank you for sharing your thoughts and that code. I will try it out.