The iPad As a Shape-Recognition System 100
An anonymous reader writes with an interesting use for the iPad: "The guys over at the Volumique blog have a different idea as to how to tackle apps for Apple's devices. They aren't just thinking about a digital activity on such devices, they are experimenting with using physical objects through Apple's multi-touch screens. Imagine being able to buy the playing pieces for a board game, but then loading up an app on your iPad for the actual board. The pieces would be recognized when placed on the iPad's screen, it would even recognize which direction they were facing. This may sound like an impossible feat unless you use a much more expensive device like Microsoft Surface, but Bertrand Duplat and Etienne Mineur at Volumique already have it working."
Possible hoax (Score:5, Interesting)
Comments attached to TFA indicate it may be a hoax, after watching the videos carefully.
Without more details or having this app actually for download it's hard to say whether real or fake. Indeed it sounds too good to be true, so it probably is.
Fake. (Score:3, Interesting)
The last choice (“jeux vidéo”) clearly selects itself a moment before the token is actually set down onto it in the first video on their blog [google.com].
Weight or shape-based? (Score:2, Interesting)
Could be a trick, of course. Could be not.
Looking at example with two figurines and iphone it seems that they have different weights - check out the bases. These seem to be button-batteries of different types and weights. In the video, however, look at 00:52 how one of responses highlights before figurine is put down.
With multiple figurines and ipad also see bases - they seem to have different radius. Detection of direction could be a bit of a hoax, maybe figurine is pressed in a specific way to help the system.
"Coloured paper" example could be explained by different amount of surface pressed against screen. Patterns are just to confuse viewers. This one is probably a trick, while the previous ones are more practical.
Comments from people who know how Apple's touchpads work (what they are able to detect) are more than welcome.
Polyhedral dice? Roleplaying applications? (Score:4, Interesting)
I thought about this quite a bit: I don't think it will be long before a major RPG system like D&D publishes an entirely paper-free edition, where the players and the DM all have touchpads. Instead of miniatures, you have avatars of your character on the game board, and certain combat actions (like determining whether a hit landed) can be automated. Basically, the "I attack this guy with my +3 sword" command would be selected on your digital character sheet, and then you just get a prompt to roll a d20 on your touchpad. All the stuff about armor adjustments, size compensation, and whatever other rules, would be calculated automatically.
I can't wait for this obvious streamlining step. I think it will most benefit rule-heavy games like Hackmaster, which are a lot more fun than D&D but rather tedious when combat situations get big. Once we get used to this, we can actually make the hit and damage modifier system even more complicated and realistic, because using weapon speed factors, armor type modifiers, armor damage, hit location determination, etc. will actually not slow down the game at all. This stuff can return the focus of the personal interaction to where it should be, which is fantasy and role-playing. As a bonus, combat rounds can be rendered and animated once this gets good enough to help with immersion.
It won't be long before the purchase of a nice touchpad will cost you less than the purchase of a set of D&D rulebooks. Then, instead of reading about character creation, it will make much more sense to just load up the character creation wizard on the touchpad and start rolling. In more advanced versions, the software will show you the first-person view of "what you see" when you kick down a door. This really is the first step.
In theory is it really that hard? (Score:2, Interesting)
It is a capacitive touch screen - right? These work by having a sheet of glass with a conductive material that when your finger (also conductive) touches it is sensed. The question then becomes - how small a "spot" can it sense and how many multi-touches at the same time? If it can sense small enough and a large enough number then a braille like symbol on the bottom of the playing pieces would be plenty to identify the item along with its direction. Some of the items they are placing on the screen *aren't* conductive so they are either faking it or have something else on it. If the latter there are a number of other applications that use capacitive dots so other than applying this to game pieces not really much new.
If the screen *can not* see like that then I can't figure out how it would work. Most bases for board games are round and there is no way to "see" which way the top of the piece is facing - round items do not have direction from their bases. One may be able to have some patterns indicate direction - say each time a piece is placed you are required to move it (maybe even a really small amount) in the direction it is facing, but that isn't inherent in just dropping the piece on the surface and it knowing where it is. Further if it is just a round base and no indentifying marks can be read there is no way to know which piece is what from just setting it on the board. Again, one could make a a system where other information is used - for instance have all the pieces start in known locations. Double tap to remove a piece from the board and every time a piece is picked up it is being "moved" to the next location it touches - basically the pieces are a mouse.
Of course those are how other things "sense" what item is on the surface and direction - the hard part is the surface.
The real issue is why? How many board games can be played on a 10" surface let alone a 4" one? At least with respect to board games (would have to think a bit about other applications) I can't see playing much of anything that way. Even with something that already uses a small board (chess) you would have to have *really* small pieces and would be hard to accurately pick them up and place them. Like many products in search of an audience if their goal is to play with some neat hardware and learn something then there is no answer to why needed past that, if they are looking to make money from this good luck - I think you are going to need it (better get a a patent so you can at least hold onto the idea until screen surfaces catch up with your idea).
Re:Possible hoax (Score:4, Interesting)
If you could arange little dot patterns of these on the underside of these objects, you would create a recognizable pattern of touches that would indicate orientation as well as position. You'd use up a number of touches though you'd need at least 3 dots and the ipad can only detect 11 touches. the ipod less. you couldn't really place a lot of objects on the screen. I guess the phone doesn't have much real estate for object placing anyway.
Functionality? (Score:2, Interesting)
I'm suspicious this is a hoax (after all, doesn't the iPad's screen detect capacitance- not visuals like patterns and shapes?) ...but even if it's real, I noticed the guy was consistently pressing down pretty hard with everything it detected. I wonder if the object's own weight isn't enough to trip whatever sensor they're using.
Re:Possible hoax (Score:3, Interesting)
Interesting. I don't know how touch screens detect the touch.
The videos do not play for me (probably a Flash issue, I'm looking forward to have that replaced with something more reliable), though the article mentions "standard board game play pieces and patterned paper". Neither are commonly conductive. The pieces shown on the images have a metal base, so should be detectable. Those bases look quite identical, so how to recognise individual pieces is a mystery to me. I would expect a dot-pattern on the bottom, or shaped bottoms. The same for the pieces that stand on their legs (let's assume they are made of some conductive material, not standard plastic).
Of course there are basically two issues here to make it work: how to detect the touch (what objects could be detected - you say must be conductive for starters), and how a touch is communicated to the internal system (as a pattern - a list of pixels that are touched, or just a mean coordinate of the centre of a touch). To be able to detect which object is placed where and facing which direction, one would have to be able to get an "image" of the touch and then do pattern recognition on that image.
And I am still wondering how many touches at the same time an iPad's touch screen can actually detect and work with. That could be another limitation.
Re:Possible hoax (Score:4, Interesting)
I've previously seen a demo done with conductive dots on the bottom of game pieces, though it was limited to only a few pieces since the iPad only detects 10 or 11 touches at a time.
It then occurred to me that there may be a hack someone figured out to detect a much higher number of touches by iterating over sets of 10. When you assume all touches are static you may be able to define a specific area of the screen to test for touches. It wouldn't work for dragging multiple objects but with it's slower refresh rate it could in theory track the positions of all objects and then compare for changes.
Further you could divide the game space up into different areas of touch sensitivity. You could have a few areas dedicated to controls which are checked more frequently and a main game area that's checked less often. There may be a slight lag but if you're checking more than 4 times a second it should feel pretty instantaneous.
It also depends on how the iOS reports touches. If there's a "new touch" event you could avoid constantly looping to check different areas of the screen.
But it doesn't seem to be too good to be true, especially with known proofs of concept already out for months.
Re:Possible hoax (Score:2, Interesting)
Actually I stumbled across this about a year ago on accident while sewing up some conductive gloves to use my iphone in the winter without freezing my fingertips off. I just never thought it would be useful for anything, though according to the linked videos I might have assumed correctly after all ;}
If you put two conductive points on the bottom of the base, with a nonconductor the right spacing between it, it will register as two fingers touching at the same time.
In my case I was going for extra surface area on a single finger, but what happened was the phone always saw one touch as being two fingers, and triggering the rotation aspect instead of dragging.
However because the spacing of the points on the finger vs the thumb of my glove were different, it measured the two touches differently in that the 'fingers' were different distances away.
In my case I had to rip that out and go with single points on each of the first finger and thumb of the glove, but I was going for normal touch usage of course.
It looks like they are doing something similar on purpose, with two touch points at different distances apart to determine which item it is. And being two points, of course you get rotation with it.
This method would limit you to one item on the screen at a time, however that is demonstrated.
It would also limit you to a small number of different items due to low multitap resolution, however that too is demonstrated with only 3-4 figures and 4 cards, but not all 8 at the same time.
This makes me want to go dig out my conductive thread and start playing again. All my work was with the iphone, but with its small screen it truly was pointless. The ipad screen would be much better for this.
Re:Fake. (Score:5, Interesting)
Fake indeed! They took down the incriminating video, but forgot to delete it [volumique.com]. Get it before it’s gone.
Re:Possible hoax (Score:2, Interesting)
I'm pretty sure he's talking about using flash to develop native iOS apps. (http://www.adobe.com/devnet/logged_in/abansod_iphone.html).
BTW, I don't think I made it clear in my original post; the reason people think its a fake is because some stuff happens on the screen before he actually places the shape/figurine on it. He claims that his finger brushed the screen.
Re:Possible hoax (Score:3, Interesting)
BTW, I don't think I made it clear in my original post; the reason people think its a fake is because some stuff happens on the screen before he actually places the shape/figurine on it. He claims that his finger brushed the screen.
Yes... check my other posts. I don’t buy that explanation – it’s pretty clear that he didn’t touch the screen.
Re:Fake. (Score:3, Interesting)