Forgot your password?
typodupeerror
Displays Input Devices Apple Games

The iPad As a Shape-Recognition System 100

Posted by timothy
from the what-have-we-here dept.
An anonymous reader writes with an interesting use for the iPad: "The guys over at the Volumique blog have a different idea as to how to tackle apps for Apple's devices. They aren't just thinking about a digital activity on such devices, they are experimenting with using physical objects through Apple's multi-touch screens. Imagine being able to buy the playing pieces for a board game, but then loading up an app on your iPad for the actual board. The pieces would be recognized when placed on the iPad's screen, it would even recognize which direction they were facing. This may sound like an impossible feat unless you use a much more expensive device like Microsoft Surface, but Bertrand Duplat and Etienne Mineur at Volumique already have it working."
This discussion has been archived. No new comments can be posted.

The iPad As a Shape-Recognition System

Comments Filter:
  • Soon the thing will take your fingerprints and surreptitiously send them off...

    • by NEDHead (1651195) on Sunday August 29, 2010 @11:21AM (#33408904)
      This will only be done in a most secure manner to verify the authenticity of the application and to certify the ownership of the download. All in the cause of guaranteeing the quality of the user experience, and with no ulterior motives. Inputting your mother's maiden name and the last 4 digits of your SS # is entirely optional, but will make available additional game features. Again, this is for your protection and is entirely secure. Play on, and don't give it another thought.
  • Possible hoax (Score:5, Interesting)

    by wvmarle (1070040) on Sunday August 29, 2010 @10:59AM (#33408812)

    Comments attached to TFA indicate it may be a hoax, after watching the videos carefully.

    Without more details or having this app actually for download it's hard to say whether real or fake. Indeed it sounds too good to be true, so it probably is.

    • Re: (Score:1, Insightful)

      by Anonymous Coward

      Since the iPad's screen will only respond to electrically conductive objects (such as your finger), and paper is used to interact with it in one of those pictures, I'm going to say there is a very high probability of this being a hoax.

      • Re:Possible hoax (Score:4, Interesting)

        by shadowrat (1069614) on Sunday August 29, 2010 @12:02PM (#33409094)
        there is a feasible way to create such a system though. there are materials that exhibit the same electrical properties as skin, and these can be used as a stylus on the idevices' screens.

        If you could arange little dot patterns of these on the underside of these objects, you would create a recognizable pattern of touches that would indicate orientation as well as position. You'd use up a number of touches though you'd need at least 3 dots and the ipad can only detect 11 touches. the ipod less. you couldn't really place a lot of objects on the screen. I guess the phone doesn't have much real estate for object placing anyway.
        • If they used a jailbreak, they would have access to the actual sensor data and write their own code there's much more data there than Apple is letting devs use. We've seen what the raw data looks like before on Slashdot, so it's more than possible, the trick for them is to get far enough ahead they can persuade Apple to open the SDK, and keep Apple from poaching the idea at the same time.

      • Re: (Score:3, Interesting)

        by wvmarle (1070040)

        Interesting. I don't know how touch screens detect the touch.

        The videos do not play for me (probably a Flash issue, I'm looking forward to have that replaced with something more reliable), though the article mentions "standard board game play pieces and patterned paper". Neither are commonly conductive. The pieces shown on the images have a metal base, so should be detectable. Those bases look quite identical, so how to recognise individual pieces is a mystery to me. I would expect a dot-pattern on the bot

        • Re:Possible hoax (Score:4, Informative)

          by PopeRatzo (965947) * on Sunday August 29, 2010 @01:40PM (#33409576) Homepage Journal

          The videos do not play for me (probably a Flash issue...

          Because maybe you're trying to watch a Flash video of this iPad on an iPad?

          • by node 3 (115640)

            The videos do not play for me (probably a Flash issue...

            Because maybe you're trying to watch a Flash video of this iPad on an iPad?

            It's clear from his post that it's not on an iPad. Surprise, Flash sucks even if you're not on a iOS device!

            • It sucks even more on an IOS device. Damn Apple and Sony for stealing other companies' acronyms.

            • by black3d (1648913)

              Elaborate?

              • by PopeRatzo (965947) *

                Elaborate?

                Not at all, it's actually pretty simple.

                • by black3d (1648913)

                  Just as I thought, you can't back up your statement. +Funny for the pun, though. ;)

                  • by PopeRatzo (965947) *

                    Just as I thought, you can't back up your statement.

                    You have me confused with the guy to whom you were replying.

                    I just jumped in to make the bad joke.

              • It doesn’t say “Posted from my iPad” at the bottom. Duh.

              • by node 3 (115640)

                Because he wouldn't say, "The videos do not play for me (probably a Flash issue, I'm looking forward to have that replaced with something more reliable)" if he were trying to view it on an iPad. You don't have "Flash issues", you just have no Flash. Also, there would be no *trying* to get the videos to play.

                • by black3d (1648913)

                  I understand, I should have specified I was after elaboration on "Flash sucks even if you're not on a iOS device!". :)

                  With the exception of low-processor workstations and mobile devices which support Flash but without the speed necessary to render tween Flash in real time, I've never really had any issues with Flash. I'm trying to determine if it's the performance or the proprietary nature or something else that you don't like?

                  I'm hoping its not a Apple fanboi stance of "since it doesn't work on iPhone/Pad/

        • by guruevi (827432)

          I think for a board game it should be quite simple. The pieces can be made conductive so that when you touch the piece, it detects you are touching a piece, they do make styluses for capacitive touch screens. If you're say, playing chess, you place the pieces in a starting position that is quite universal. From there on, all you have to do is detect the touches (from where to where) and you'll be able to 'know' where each piece is at any given point in time.

      • by baxissimo (135512)
        Lots of things conduct besides fingers. Could be the paper was treated with dots of some conductive ink. You could use the relative locations of a pattern of conductive dots (which the iPad would interpret as multiple "fingers") to encode a bit of information that tells you what a particular object is. I haven't seen the actual video, but within limits, certainly seems feasible to me that you could do Surface-like things on an iPad. One main limitation will be the maximum number of contacts that the i
      • Ever take apart a remote control to clean the backs of the rubber buttons and corresponding contact points on the PCB? Nice soft conducting material there... I think it's a carbon embedded in the rubber. Whatever it is, it would make a conductive game piece base and could have a unique pattern in it.
    • Re:Possible hoax (Score:4, Interesting)

      by CODiNE (27417) on Sunday August 29, 2010 @01:55PM (#33409670) Homepage

      I've previously seen a demo done with conductive dots on the bottom of game pieces, though it was limited to only a few pieces since the iPad only detects 10 or 11 touches at a time.

      It then occurred to me that there may be a hack someone figured out to detect a much higher number of touches by iterating over sets of 10. When you assume all touches are static you may be able to define a specific area of the screen to test for touches. It wouldn't work for dragging multiple objects but with it's slower refresh rate it could in theory track the positions of all objects and then compare for changes.

      Further you could divide the game space up into different areas of touch sensitivity. You could have a few areas dedicated to controls which are checked more frequently and a main game area that's checked less often. There may be a slight lag but if you're checking more than 4 times a second it should feel pretty instantaneous.

      It also depends on how the iOS reports touches. If there's a "new touch" event you could avoid constantly looping to check different areas of the screen.

      But it doesn't seem to be too good to be true, especially with known proofs of concept already out for months.

    • Re: (Score:2, Interesting)

      by dissy (172727)

      Actually I stumbled across this about a year ago on accident while sewing up some conductive gloves to use my iphone in the winter without freezing my fingertips off. I just never thought it would be useful for anything, though according to the linked videos I might have assumed correctly after all ;}

      If you put two conductive points on the bottom of the base, with a nonconductor the right spacing between it, it will register as two fingers touching at the same time.

      In my case I was going for extra surface

    • Re: (Score:2, Insightful)

      by Tharsman (1364603)

      It's not impossible, and like always, this slashdot title is the one lying. The hard pieces could have multiple spots (not just a shape) the screen would recognize as multi touches and then it's up to the app to recognize the pattern.

      You see, the article does not use the word "shape" anywhere, they just say "object recognition" and that's a world of difference.

      It is very likely they have come up with some sort of sticker with pre-defined touchspots and either pre-programmed these patterns on their app c

      • by node 3 (115640)

        Three dots, sufficiently not in a row, are called a triangle. Look up in the sky tonight (northern hemisphere) and you'll see what's called the Summer Triangle.

        These three dots make up a triangle, these three dots a square, etc. Claiming that it's a lie to say these are shapes is a bit much.

        • by Tharsman (1364603)

          Did you flunk Geomotry? Three dots are three dots. They CAN represent a triangle as long as you properly define them as 3 line segments, but by themselves, the 3 dots, a triangle are not.

          In the end, the title is a lie because not only there is no mention of such technology in the article, but also because the iPad can't detect shapes. Put a square material that the iPad can recognize on it's surface and the OS just will notify one XY feedback, not the covered area (reason why people without enough foresig

          • by node 3 (115640)

            Did you flunk Geomotry?

            Really? No, I didn't flunk geometry.

            Three dots are three dots. They CAN represent a triangle as long as you properly define them as 3 line segments, but by themselves, the 3 dots, a triangle are not.

            No, you do not need to actually connect the dots to make a triangle.

            The problem is that when someone says, "makes a triangle" or "makes a shape", they don't always mean you have actually connected the dots. You may be surprised to learn that not everyone speaks solely in mathematical terms.

        • Three dots, sufficiently not in a row, are called a triangle.

          Not if you do not draw lines between them. But lines do not enter into touch detection in iOS, all you ever have are points.

          • by node 3 (115640)

            Three dots, sufficiently not in a row, are called a triangle.

            Not if you do not draw lines between them. But lines do not enter into touch detection in iOS, all you ever have are points.

            This is only true if the author intends the term to be taken in a strict mathematical sense. The word "shape" is not solely mathematical. Neither is triangle. Hence the Summer Triangle.

            Or are you saying that astronomers have actually connected the three stars in question? That would be quite a feat!

            • Hence the Summer Triangle.

              Or are you saying that astronomers have actually connected the three stars in question?

              "The Summer Triangle is an astronomical asterism involving an imaginary triangle drawn on the northern hemisphere's celestial sphere,"

              An imaginary DRAWN triangle still involves lines so yes, that is exactly what they have done. The points in question for detection are a pattern, not a shape. You are also making an assumption without proof that the points are not along a line, when that would ma

              • by node 3 (115640)

                An imaginary DRAWN triangle still involves lines so yes, that is exactly what they have done.

                Do you know what "imaginary" means? It means that no, that's *NOT* what they've done, they've just imagined it being done.

                And in that exact same way, there's nothing wrong with calling three dots a "triangle", because the speaker is similarly conveying an imaginary shape.

                • And in that exact same way, there's nothing wrong with calling three dots a "triangle", because the speaker is similarly conveying an imaginary shape.

                  As the original poster noted, there is no shape - imaginary or otherwise. That's where you fail to grasp what is going on. It's a pattern, not a shape...

    • by joek1010 (980753)

      Here's the guy's response to a comment about this on his website:

      Non tu ne rêves pas, mais j’ai juste appuyé avec mon doigt au moment de prendre le pion et cela à créé un contact. La démo est développée en Flash et marche vraiment très bien. Nous sommes en train de la redévelopper en Objective C (merci Steve;-).

      It says: No, your not dreaming. I just pressed (brushed?) my finger at the moment when I picked up the piece and that created a contact. The de

      • Re: (Score:3, Informative)

        by clone53421 (1310749)

        They got Flash to work on the iPad?

        • Re: (Score:2, Interesting)

          by joek1010 (980753)

          I'm pretty sure he's talking about using flash to develop native iOS apps. (http://www.adobe.com/devnet/logged_in/abansod_iphone.html).

          BTW, I don't think I made it clear in my original post; the reason people think its a fake is because some stuff happens on the screen before he actually places the shape/figurine on it. He claims that his finger brushed the screen.

          • Re: (Score:3, Interesting)

            by clone53421 (1310749)

            BTW, I don't think I made it clear in my original post; the reason people think its a fake is because some stuff happens on the screen before he actually places the shape/figurine on it. He claims that his finger brushed the screen.

            Yes... check my other posts. I don’t buy that explanation – it’s pretty clear that he didn’t touch the screen.

  • Fake. (Score:3, Interesting)

    by clone53421 (1310749) on Sunday August 29, 2010 @11:01AM (#33408824) Journal

    The last choice (“jeux vidéo”) clearly selects itself a moment before the token is actually set down onto it in the first video on their blog [google.com].

  • Surface (Score:3, Informative)

    by Zouden (232738) on Sunday August 29, 2010 @11:02AM (#33408828)

    Microsoft Surface is also much larger (the size of a coffee table or so) where it might make sense to put objects on it. Board gaming on a Surface would be quite an experience, but with an iPad you'd be covering most of the display unless you stuck to just a few playing pieces. You also have to fit the entire board on the screen, as it won't be able to scroll with pieces on it. The example given in the article, Monopoly, just wouldn't wouldn't work at all.

    • by PopeRatzo (965947) *

      Monopoly, just wouldn't wouldn't work at all.

      And take it from me, don't try to play mumblety-peg on your iPad.

    • by Jesus_666 (702802)
      Hmm. Surface projecting on a Battletech playing field + knowledge of playing field topography + 'mech recognition (perhaps via small patterned labels on the 'mechs) + Battletech rules = realtime display of movement ranges, to-hit chances etc.

      Or, more involved, use hexes with blank tops and project the terrain onto them so you can even use the optional rules for fires and cleaning out terrain.
  • by Moskit (32486)

    Could be a trick, of course. Could be not.

    Looking at example with two figurines and iphone it seems that they have different weights - check out the bases. These seem to be button-batteries of different types and weights. In the video, however, look at 00:52 how one of responses highlights before figurine is put down.

    With multiple figurines and ipad also see bases - they seem to have different radius. Detection of direction could be a bit of a hoax, maybe figurine is pressed in a specific way to help the sy

    • by Verteiron (224042)

      Interestingly enough the first video is now only 46 seconds long. Never saw the pre-selecting piece.

    • Just speculation but a conductive surface registers as a touch, so different size magnets could register as different size touches. Maybe the cards have some sort of tape on the bottom that consists of patches of conductive surface mixed with non-conductive surface which would register as a pattern of touches, sort of like braille.

      As to the allegation of cheating on the one video, the admin claims (in french) in the comments he grazed the screen with his finger and this is why the touch seemed to register b

    • by anethema (99553) on Sunday August 29, 2010 @12:42PM (#33409274) Homepage
      Certainly not weight based. The touchpads are capacitive sensing touchpads. You generally need conductive materials, and it usually calibrated for the range of human fingers. Some accessories have been made that transmit the capacitance of your finger through a spot in a glove etc so you can use it with gloves on.

      As for the chess pieces etc I imagine they could organize the conductive material in the chess piece to indicate direction, but for stuff like the coloured paper, no f'n way. Paper will certainly not trigger any sort of touch response, and there is absolutely no way to sense color.

      I personally have a feeling the whole source material is a hoax.
      • by AC-x (735297)

        Those cards are definitely fake, if you watch his thumb he's obviously putting pressure down on the home button.

        The figure recognition could be based on size, but then I didn't see anything on the bottom that would allow it to sense which direction it was pointing. If there was 2 pads on the bottom arranged like oO then it would have at least been vaguely believable.

        • Some of the other videos on their blog [volumique.com] have been goofy-looking / doubtfully credible too.

          For instance this [volumique.com] – apparently supposed to be using 2D barcodes to identify video clips for an interactive “choose your adventure” style flipbook, but – the exact same card is “scanned” in two demonstrations and two different video clips play.

          Or this [volumique.com] ... notice that in every shot where his hand is in the video frame, you can tell that the video quickly does a blend transition from one

  • Device with large multi-touchscreen can be used as a large multi-touchscreen! How about we just gut it and use the screen as a sensor? I know it would leave Apple out, but I'm okay with that.
    • Maybe we could even get a pen for it and make the combination pressure-sensitive. Imagine what it could do for artists and drafters. I'm going to start a company doing this. I'll call it "Wacom".

  • by Dr. Spork (142693) on Sunday August 29, 2010 @11:55AM (#33409058)

    I thought about this quite a bit: I don't think it will be long before a major RPG system like D&D publishes an entirely paper-free edition, where the players and the DM all have touchpads. Instead of miniatures, you have avatars of your character on the game board, and certain combat actions (like determining whether a hit landed) can be automated. Basically, the "I attack this guy with my +3 sword" command would be selected on your digital character sheet, and then you just get a prompt to roll a d20 on your touchpad. All the stuff about armor adjustments, size compensation, and whatever other rules, would be calculated automatically.

    I can't wait for this obvious streamlining step. I think it will most benefit rule-heavy games like Hackmaster, which are a lot more fun than D&D but rather tedious when combat situations get big. Once we get used to this, we can actually make the hit and damage modifier system even more complicated and realistic, because using weapon speed factors, armor type modifiers, armor damage, hit location determination, etc. will actually not slow down the game at all. This stuff can return the focus of the personal interaction to where it should be, which is fantasy and role-playing. As a bonus, combat rounds can be rendered and animated once this gets good enough to help with immersion.

    It won't be long before the purchase of a nice touchpad will cost you less than the purchase of a set of D&D rulebooks. Then, instead of reading about character creation, it will make much more sense to just load up the character creation wizard on the touchpad and start rolling. In more advanced versions, the software will show you the first-person view of "what you see" when you kick down a door. This really is the first step.

    • by NoStrings (622372)
      There are already lots of games like this - albeit without the touch screen. Look at Neverwinter Nights, for instance. To me, a system like this would take away from the fun of a pen & paper RPG. It's nice to unplug for a while, sit down with some friends face to face, and roll some dice. I think that it would be nice to have a big touch screen for the main map, but once you let the system automate the rolling of the dice, it basically becomes a computer RPG.
      • by Dr. Spork (142693)
        But that's what I'm talking about. You'd be around a table with friends. It's just that your character sheet would be a touchpad. You'd roll your dice on it and it would do some of the tedious number-crunching so that you could get on with the face-to-face interaction. Also, your rulebooks would be software in the touchpad.
        • by PopeRatzo (965947) *

          You'd be around a table with friends. It's just that your character sheet would be a touchpad. You'd roll your dice on it

          There's a microprocessor in that thing. Why would you need dice at all? And wouldn't the dice just bounce right off the iPad's screen?

          The nice thing about pad and paper is that when you get too drunk to do the calculations or see the dots on the dice you know you're too drunk to play the game and it's time to go home.

          And please don't throw hard plastic dice onto the screen of my iPad.

          • by dissy (172727)

            I'm not sure if you are making a joke and I missed it, but your first sentence implies you truly misunderstood.

            You'd be around a table with friends. It's just that your character sheet would be a touchpad. You'd roll your dice on it

            There's a microprocessor in that thing. Why would you need dice at all? And wouldn't the dice just bounce right off the iPad's screen?

            Sorta hard to use the microprocessor to roll dice if you dont run software on the thing ;}

            He wasn't referring to physically rolling the dice, literally on the ipad.
            But having the software do it for you, thus the dice rolls are generated on the ipad in software.

            So, is the woosh on me or you for this one?

            • by PopeRatzo (965947) *

              So, is the woosh on me or you for this one?

              If you look up "woosh" in the dictionary, there's a picture of me.

              But I thought since this entire discussion is about using touchscreens to detect positions of game pieces sitting upon it, that the dots on the dice would be conductive, allowing the touch screen to determine the value of the roll.

              I was kind of making a joke, because the guy said there would be a touchscreen and you'd "roll the dice on it". I couldn't really tell if he actually meant "roll the dice

              • by dissy (172727)

                I suppose I too shouldn't assume what the original poster meant.

                I can definitely see with the topic at hand how you would read that, and for all I know that's what he meant.

                Just thinking about his (fairly good) idea of using the pad as a character sheet however, it would be most wise to let it do all the math and game mechanics for you, including random numbers.

                In fact I have an iPhone app that does just that, though it is more for novelty sake as it does the sounds and everything (Dice Bag i think it is ca

            • actually, with this tech, you could make a set of dice that really would "roll" on the touchscreen. It would be a neat idea, of course like the Wii fiasco, people would smash their damn screens tossing dice that are just a few grams.

        • by Gulthek (12570)

          We've got that now. d20srd + myth-weavers = rules and character sheets on the iPad

          I could even tie in a dice rolling application that tracks statuses and their corresponding effects on the dice, but I still prefer physical dice.

      • by PopeRatzo (965947) *

        It's nice to unplug for a while, sit down with some friends face to face, and roll some dice.

        I don't know about you, but I've got some friends that just hanging out with them is rolling the dice.

    • Already in the works (Score:3, Informative)

      by RingDev (879105)

      You mean an RPG game like this: http://www.youtube.com/watch?v=n94E3IeBquY [youtube.com]

      And a cheap DIY Surface like this: http://www.maximumpc.com/article/features/maximum_pc_builds_a_multitouch_surface_computer?page=0,0 [maximumpc.com]

      -Rick

      • by Dr. Spork (142693)
        Both of those links are very cool, thank you! In the first, I didn't like the virtual dice, and the stupid dramatic delays, but the concept seems good. I was picturing this as more of an app on touchpads that people would own for other reasons, not one giant table-sized touchpad. But then again, I can imagine that there might be some other cool apps for a table-sized touchpad, like those in your second link. Since table-sized HDTVs are becoming affordable, maybe this really is the way to go. That multitouch
    • by Jesus_666 (702802)
      Unlikely. The appeal of pen-and-paper RPGs is the immense freedom the players have to do whatever they want, usually involving stuff not covered by the rules. Also, most groups have their own house rules. The only thing that would reliably work yould be a dumb display with the GM doing all the work. You can abstract out checks (unless houseruled) but everything else is a bit too volatile.
  • It is a capacitive touch screen - right? These work by having a sheet of glass with a conductive material that when your finger (also conductive) touches it is sensed. The question then becomes - how small a "spot" can it sense and how many multi-touches at the same time? If it can sense small enough and a large enough number then a braille like symbol on the bottom of the playing pieces would be plenty to identify the item along with its direction. Some of the items they are placing on the screen *aren't*

    • by tibit (1762298)

      I think that touchscreens aspire to the same problems as eye-tracker based pointer control: in both the control and visual inputs are mutually exclusive. If you use a touch screen, you can't see what you're pointing at. With an eye-tracker control, you can't see anything but what you're pointing at. Both equally bad scenarios.

      I think that eye motion and touch screens were first imagined to be input devices by SF writers who couldn't foresee all of the shortcomings of those input methods.

      Give me a separate k

  • Functionality? (Score:2, Interesting)

    I'm suspicious this is a hoax (after all, doesn't the iPad's screen detect capacitance- not visuals like patterns and shapes?) ...but even if it's real, I noticed the guy was consistently pressing down pretty hard with everything it detected. I wonder if the object's own weight isn't enough to trip whatever sensor they're using.

  • The ReacTable [reactable.com]...Bjork had one of these on her most recent US tour. Lots of fun to watch in action.

  • NOT FAKE ! (Score:1, Informative)

    by Anonymous Coward

    it's not fake. It can only work for one object at a time though. It's pretty simpe... there are two contact points on each object that simulate the touch of a finger. the distance between the points indicates the type of object. and the angle between the points indicates rotation of course ! But they can't scale this up... from what I can tell, one object at a time is a hard limit. But this is a software limitation probably, not a hardware limitation.

    • I don't see why you would be limited to one object. If you know the radius of the object and you allow a margin around the edges where you won't put any dots, you can have a variable distance between your two dots (like you said) that differentiates one from another. The margin would have to be big enough, though, to keep your dots from intersecting with dots of other objects. Other than that, just keep it under your max number of touches (which I believe is in the ballpark of 10 touches) and you should

  • by Anonymous Coward

    On the second demo, notice that the thumb on the bottom part of the card is ALWAYS right over the home button? And notice that they don't show the app quitting? I suspect they're just tapping the home button, which the app is capturing and showing a static image in a fixed order. FAKE.

    • I replied on their site with this exact post and guess what? It's moderated and they nuked my post. FAKE for sure.
    • by AC-x (735297)

      Watch his thumb, the end of his nail goes white which means he's applying pressure, and his thumb is right over the home button...

      These guys are amateurs, if they're going to fake it they should have used thinner paper so their finger press would register through it (I've tried it on my htc hero and the touch screen does work through a sheet of paper)

2.4 statute miles of surgical tubing at Yale U. = 1 I.V.League

Working...