Five tips for tuning and optimizing the Leap Motion Controller. Leap Motion - how to set it up to work Download the app for leap motion

Motion capture controller for PC/Mac

One of the most high-profile tech startups of the past year, and one of the five iconic gadgets of the year according to the magazine, is the Leap Motion motion capture controller. Last summer, users who supported the project on Kickstarter began to receive long-awaited controllers. It's time to brush off ambitions and take a sober look at the device.

And first of all, strictly following the saying that it is better to see once, we offer to evaluate the work of Leap Motion on video:

Design

In the box, along with the gadget, there is a booklet and a pair of Micro-USB 3.0 cables. One long, for connection with system block(slightly less than 1.5 m) and one short, half-meter, for a laptop.

The appearance of the controller itself is uncomplicated. Two infrared eyes are visible through the upper glossy surface during operation. The slot on the side panel glows green when the device is turned on.

The USB port is on the left. Although the package contains cables of the third version, the controller works fine with the second.

The underside of the Leap Motion is rubberized for better grip on the table.

With dimensions of 80 × 30 × 11 mm, the weight of the gadget is about 50 g.

Usage

Let's start with the simplest - operating system management. We download the Touchless application through the browser (there are versions for Mac OS and Windows) and run it using Leap Motion Airspace.

Touchless will remind you of itself with a tray icon. Operating system management is activated via the context menu.

The application allows you to apply basic set gestures, identical to what we are used to in touch screen devices. The touch indicator is a green circle.

In principle, Touchless does not offer anything special. Left click - with a short press on the virtual surface, right - with a long press. Horizontal and vertical scrolling. Zoom using two fingers of one or both hands. On this, perhaps, everything.

The free Touchless utility is recommended by HP for OS management. In addition to it, a dozen and a half similar applications are available in the store, placed in a separate category. And at the time of writing, two-thirds of them were paid. In general, the Leap Motion store looks too monetized for its level of development.

In the gadget settings, you can set the preferred height at which the sensor will read movements.

In addition, there are three options to balance the speed and accuracy of gesture processing. The set of settings, as we can see, is small, but it's good that the manufacturer allows you to change at least something.

conclusions

More than a year has passed since the first Leap Motion samples were sent out. Despite the convenient form factor and interesting concept, in the long term, the new controller has not yet justified the hopes placed on it and monetized much faster than it develops. Free Apps They offer mostly basic functionality and rarely claim to be more than a banal demonstration of the controller's capabilities. The store has add-ons for professional tools, Photoshop and Autodesk Maya, but this is negligible. In the current state of the “gadget in itself”, Leap Motion is likely to appeal only to curious enthusiasts who expect, first of all, to play with the gadget. And let's hope the developers aren't burning Kickstarter revenue on a small tropical island, but partnering with big companies like HP will prove beneficial to all stakeholders.

I remember how last year I saw and did not believe that this is not a prototype, but a completely completed product, the release of which will take place within six months. Somehow, the pictures shown by the developer looked too fantastic, suggesting thoughts about the Minority Report film. But the miracle did not happen, and at first the release date was postponed to, and then. But the main thing, as they say, is the result, and it is very, very good. Yes, it’s not bad there - this is a really amazing thing, which I will talk about in this article.

Concept

The idea behind Leap Motion is to make the user interface more intuitive. And it does not matter whether you are working in the operating system or playing a game. And what could be more intuitive than gestures? Remember how cheerfully Tom Cruise worked with the Agency database in the film Minority Report:

Impressive, isn't it? That's how easy the data is chasing around the screen, combined into separate clusters, compared. With a keyboard and mouse, it would obviously take much longer to poke around.

« But this is a matter of the distant future and, in general, an invention of a science fiction writer!”, you say. And look around - most of the technologies in modern world came out of fairy tales and fantastic works: moving through the air at great speed, communication at a distance using a miniature device, which is a powerful computer, easily fits in the palm of your hand, the Internet ... there are a lot of examples.

So Leap Motion has become a kind of dream come true Philip Dick who wrote the novel Special opinion back in 1956:

It is no less impressive than the episode from the film shown above. And, you know, this thing really works! It may not be as smooth as it is demonstrated in the video, but the feeling of working with Leap Motion is difficult to convey in words, although I will try to do it.

Small device in a big box

Given the dimensions of the controller, not much larger than those of a lighter, it comes in just the same huge box, although that one can easily fit in the palm of your hand. All in all, the packaging is great.

Dust jacket on top, under it is a box made of very hard and high-quality cardboard. You lift the lid and you are invited to plunge into a whole new world:

And here is the hero of the review himself, freely located in a plastic bath:

In addition to it, there are a couple more cables and a small instruction in the box:

By the way, pay attention to cables. The length of one is 60 cm, the second is 150 cm, but the connector is more interesting:

This is not a proprietary connector, but a standard one. micro USB 3.0. But, to be honest, I did not understand why it was necessary to use this option if the device itself works via USB 2.0. At least that's what it says on the official website. The usual microUSB connector is more compact, and it is easier to get the appropriate cable if the complete one becomes unusable.

Well, okay, what we have, we have. Let's take a closer look at the device itself.

The Leap Motion, though small, is heavy and tightly packed. The controller is made in an aluminum case with a glossy plastic panel on top and a rubber one on the bottom:

The accessory is assembled perfectly - a monolith: it does not creak, does not play, nothing moves. I liked the solution with a rubber bottom - due to it, the gadget stands securely on almost any flat surface.

Of the additional elements on the case, there is only a microUSB 3.0 connector and a status indicator:

Imagining myself the smartest and able to figure everything out without instructions, I decided to immediately connect the controller to the Mac and act according to the circumstances, without bothering with pieces of paper. Plugged it in and… nothing happened. Instructions are for cowards? No, friends. Still, it is better to read them, which saves both time and nerves.

How to make Leap Motion friends with a computer

But all that had to be done was to go to the official website of the project. The creator of the accessory strongly encourages this as soon as you open the box and take the gadget out. On protective sticker above it is clearly written, they say, " Activate at leapmotion.com/setup". Following the link, I received a suggestion to download the appropriate drivers:

By the way, the package was downloaded very weighty, as much as 80 MB, and after installation it takes up more than 200 MB:

Well, I installed the drivers, software, and after connecting to the computer, the small box immediately came to life: the status LED turned green, and red dots appeared on top - this is an IR emitter:

There was action on the screen, too, with the software not only prompting me to plug in the gadget and position it appropriately, but also clean the Leap Motion's top panel of fingerprints to make the device work better. I don’t know where it found prints there, I took out the device very carefully, didn’t touch the top panel, but still wiped it, brushing off a few dust particles:

All right, now I'll touch the magic! But no, you still need to launch the Airspace companion app and register with your local app store:

Registered, launched Airspace and, finally, you can touch the future. Don't rush though. Demo applications will load first, and they take a decent amount - at least tens of megabytes, but usually well over a hundred.

Well, while it is loading, it is worth going over the settings of the accessory, there are not many of them.

In the tab with basic settings you can allow or prohibit the transmission of information from the sensor to web applications and programs running in the background, as well as adjust the height at which tracking is carried out:

In the tab tracking the priority of the sensor is configured (speed, accuracy, or balanced operation) and the scheme of its operation (whether to automatically track the orientation of the hands in the air and whether to filter out unnecessary objects in the Leap Motion coverage area, such as shoulders and head).

Well, all the demo programs have loaded, it's time to test the gadget in action.

Video review

Hand passes

When the packaged Leap Motion got to the editors, it did not cause any special emotions. We waited too long and somehow burned out, probably. Even in the process of setting up the device, I personally did not experience particularly strong emotions.

But it was worth launching the first application and seeing the device in action, when without touching the keyboard, mouse or screen you do some hitherto incredible things, like rotating the DNA molecule just like in Minority Report, or scaring away flocks of fish with your lungs with the movement of the palm, then the wild delight and the feeling that the world is changing right before your eyes returned.

controller really very accurate tracks palms, fingers and their movement. True, you still need to get used to it. First, do not make too sudden movements. The performance of Leap Motion is high, but don't expect miracles. Moreover, at sudden movements often the hands fall out of the range of the sensor.

Second important point- this is the definition of the most convenient tracking height, which is configured in the options, I mentioned this above. During testing, the standard value of 20 cm turned out to be the most convenient, but for the most part I interacted with the gadget while standing. If you are sitting, then you should slightly lower the height, or use automatic tracking.

Concerning applications, then for Leap Motion there are already a lot of them, both paid (prices are approximately at the level of the Apple App Store) and free. In fact, the company that created the device opened its own application store, which many third-party developers have already successfully mastered. Fortunately, the SDK has been distributed for more than a year.

There are games, there are just cool programs that demonstrate the capabilities of the controller, there are applications for creating music, learning, etc. There is software for both Windows and OS X.

In general, for the most part, this is still a toy, but an incredible toy, causing a storm of emotions in the soul. I was happy as a child when virtuality reacted to my hand movements in this reality - when ripples appeared on the water from touch, and the crown of a tree with luminous leaves moved, when you control a virtual racket in a 3D arkanoid and hit the ball as if you were holding it in your hands a real racket... it's all very addictive.

And it is worth trying to play with Leap Motion for a child, and you will not drag him away from the computer for a long time. Moreover, the children do not need to explain anything, they grasp everything on the fly, and now the 3-year-old kid briskly cuts the ropes and feeds Omnoma with candies.

Leap Motion has a great future. Let it be more of a toy for now than a serious tool or replacement for a mouse and touchpad, but it really works. Moreover, the device was supported by the developers and this is the most important thing. Indeed, without software, even the coolest gadget will be just a pile of dead "iron, plastic and silicon." Software is the soul of the device, and Leap Motion's is broad, kind, and promising.

The issue price is about 5000 rubles.

website I remember seeing a Leap Motion demo last year and not believing it wasn't a prototype, but a complete product that would be released within six months. Somehow, the pictures shown by the developer looked too fantastic, suggesting thoughts about the Minority Report film. But the miracle did not happen, and at first the release date was postponed by four...

The Leap Motion Controller was named one of the top 10 devices of the year by Time magazine. This device belongs to the glorious family of new generation wireless controllers such as the Wii Remote, PlayStation Move, but its closest relative is the Xbox Kinect. Unlike the latter, Leap Motion reacts exclusively to the movements of the hands, it detects even the fastest movements of the hands and fingers 200 times more accurately. This device brings us even closer to the real virtual reality- to create a natural interface between man and machine. Hurrah, comrades!

Leap Motion Controller

After the release of the Kinect sensor, other contactless control devices began to appear in the wake of its success. Kinect served as the basis for the growth and development of the market for such devices: investors saw the prospect and understood the meaning of investing in gesture control devices. However, the most significant and successful was the Leap Motion Controller. Like its progenitor, the latter is based on motion capture technology. This device connects to USB port and does not exceed the size of a pair of folded flash drives. On the technical side, to capture the projection of user hands in space, the Leap device uses two optical sensors(cameras) and an infrared light source (the developers do not exclude that the number of cameras may be changed in future versions of the device). The device is placed with a working surface up next to the screen to create the feeling that the objects on the screen are operated with the help of hands. After connecting the device, a virtual inverted pyramid is formed above it with the central vertex in the device. The most effective range extends from 25mm to 600mm above the controller with a 150 degree field of view. In the area of ​​this pyramid, Leap Motion “sees” all movements and sends them to software, which converts data and signals into coordinates and messages. The software is able to recognize both simple gestures (virtual touches, clicks) and complex long movements: scaling, moving, rotating, drawing various geometric shapes. Thus, the device itself does not perform any calculations and transformations, leaving everything at the mercy of the host software, which, by removing image noise, builds models of hands and fingers - pointers. Having the origin in the center of the device, Leap Device interprets the coordinate axes as follows: negative X is located to the left of the device, respectively, positive - to the right. The Y coordinate grows upwards and has no negative values, since Leap "sees" objects starting from 25 mm above itself. Positive Z is towards the user, while negative Z is towards the screen.

Leap Motion SDK

The Leap Motion SDK is developing surprisingly rapidly, and new versions are released with enviable regularity: in a relatively short history of its existence, a full-fledged second version of the tools has already appeared, as well as its modifications. More precisely, the mods are still in beta, and we will use the latest version of the SDK at the time of writing, since each a new version provides visible improvements - additional tracking capabilities for the skeleton ("bones" of the hands). As expected, Leap Motion SDK works on all common platforms: Windows NT, OS X, Linux. Since lately I have to work more on a Mac (and I'm editing this article on an EEE PC with Win XP, and I'm fine. - Approx. ed.), In the future, my story (with some reservations) will relate to this particular operating system. If you are not friends with her, do not despair, because the Leap Motion SDK is cross-platform, and you can easily adapt the information obtained from this article for any supported operating system.

Ready to rock!

To start working with the Leap Motion controller, after registering on the device manufacturer's website, download the LeapDeveloperKit_2.1.1+21671_mac.tar archive from the Downloads section. After unpacking it, you will find a folder containing the Leap_Motion_Installer_skeleton-release_public_mac_x64_2.1.1+21671_ah1704.dmg bundle (disc image for OS X) containing drivers for the device, as well as demo applications. Next to the bundle will be the LeapSDK directory, which includes all the necessary libraries and APIs for developing applications that work with the Leap Motion device. In addition, this folder contains documentation and samples. In addition to demo applications, the bundle contains Airspace Home, a kind of client for the Leap Motion app store - you can upload your applications to it and sell them, like on other digital distribution sites. The main difference between the second version of the SDK and the first one is new system tracking the "skeleton" of the upper limbs. It includes processing additional information about the bones of the hands and fingers, the ability to predict the location of bones invisible to the device and the construction of hand models in those conditions when the limbs are not completely visible.

First, install the contents of the bundle (I'm sure under Windows it has the same name, only with the .exe extension). The installer itself, located inside the image, is called Leap Motion.pkg, it starts the installation process of all of the above.

Rice. 2. Program installation

After the installation of the software for Leap Motion is completed, the driver will automatically start, which will “settle” in the form of a demon in the menu bar (top right). Three new applications will appear in Programs: the driver itself, the Leap Motion Orientation demo program (I recommend starting with it), and Airspace. If the controller has not been connected before, it's time to do it. The icon (in the menu bar) will be highlighted in green. Clicking on it will open a menu containing five items. The first item Launch Airspace launches the window client of the same name. By default, it contains seven demo apps and two links leading to the Airspace Store and the developer community. Each of the demos reveals the possibilities of Leap Motion.

The next menu item - Settings opens a window for configuring the device. This window includes four tabs. On the Generals page, the main settings are made: allow or prohibit the device from interacting with web applications that support Leap Motion (looking ahead, I note that this possibility is present, and HTML5 + JavaScript is used for this), enable or disable the ability to receive signals from the device to applications , working in the background, automatically transfer device statistics, enable (if necessary) switch to power saving mode; adjust the smallest height above the device at which it "sees" hands and fingers (pointers); to agree to automatic update. There are two options on the Tracking page related to configuring the device's "tracking" capability. The next tab is dedicated to diagnosing and troubleshooting, there are functions for viewing the software log, a diagnostic visualizer, recalibrating the device and resetting to default settings. The last tab simply reports information about the device and the software that serves it. By clicking on the Visualiser item, a demonstrator opens, in which you can see how the device “sees” the limbs. That is, if you move your hands over the active area of ​​the device, the application will display them in virtual space. The Pause Tracking button pauses tracking, Quite - cuts down the demon.

Once the Leap Motion software is installed, you can install the developer tools. At the same time, I assume that you have the latest versions of the operating system and development tools (Xcode) installed. As I said above, after unpacking the archive, the folder with the SDK is located next to the installation bundle. This folder contains documentation, examples, header and object files for all officially supported languages. Initially, the Leap Motion SDK was written in C ++, but thanks to SWIG, it has support for many common compiled and interpreted languages, including C # (together with the .NET and Mono frameworks plus the Unity 3D engine), Objective-C, Java, Python, JavaScript. SWIG, being free tool open source, acts as a glue code generator between C++ and other languages. For our developments, we will take C ++ as the most native. The client computer and the controller interact via a TCP connection, which opens ports 6437, 6438, 6439 - for the device to work correctly, it is necessary to ensure that they are not blocked by the firewall. The Leap Motion SDK allows you to develop two types of applications: those that support the native interface (client applications) and the WebSockets interface (web applications that run in a browser environment). The first for work (receiving data from the controller) use a dynamic library - specific for a particular operating system, it connects to the device and provides a service to the top level. While the latter receive data through the localhost WebSockets server in the form of JSON format messages. In this case, JavaScript + open source LeapJS add-on is used, and to control the device, the application can send configuration messages through the WebSockets server back to the device.

Leap Motion coding

Today we will focus on native applications for OS X, but thanks to the cross-platform tools, you can easily convert our programs for another supported operating system. We will not develop a console application that shows the coordinates passed by the controller, this is boring. We'll dive straight into some serious code and write an application that displays a graphical representation.

Visualization

The Leap Motion SDK provides wonderful facilities for getting data from the controller, but it doesn't have anything at all for graphics output. Therefore, our path lies through the use of additional tools. To display graphics from a native application under OS X, you need to use OpenGL. Sadness emanates from this idea: the level is too low, no article is enough, and in general you can fall asleep. Therefore, we will use the setting over OpenGL. Of the widest range of such libraries, I chose Cinder. Cinder is a set of open source libraries for image processing, graphics, sound, computational geometry. As I said above, Cinder is cross-platform, and the same code will work not only on desktop platforms, but also on Apple smartphones and tablets. In the future, developers are going to expand the range of supported hardware and software platforms. In addition, to generate a blank for a new project, the Cinder distribution includes the TinderBox utility, with which you can create a project with support for OpenGL, DirectX, CocoaView (OpenGL), each of these blanks can contain support for the Box 2D physics engine, the Cairo rendering library, the FMOD audio library , the OpenCV computer vision library. For Apple devices, you can generate a blank, where geolocation and motion managers will be used, using standard frameworks (Core Location, Core Motion). All this can be easily included in the project at the stage of its creation using the GUI interface. In addition, the project can be generated for a specific programming environment and operating environment: Xcode (Mac), Xcode (iOS), VC 12/13 (WinRT). Corollary: we have more than an API library, it all feels like a cross-platform game engine! You can also create a local Git repository right away. In my humble opinion, Cinder will soon become the best cross-platform solution, even compared to Qt. Since boost is heavily used in Cinder, it's a good idea to update it to latest version. Open your favorite console and first install the management system for outdated (in the harsh eyes of Apple) Homebrew packages:

Ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Then install boost 1.55 from this system: brew install boost. To work directly with Cinder, it is enough to download and unpack it, and to generate a project, use the TinderBox utility located in the tools subfolder.

Hands, fingers, space management

So, to warm up, let's create an application that displays what the sensor sees in the window. If you read my articles about Kinect, then you can remember that we started there the same way, we will consider it a tradition. The template from TinderBox for OpenGL is perfect for us, we just need to add Leap Motion support to it. To do this, drag two files from the include subdirectory of the previously unpacked LeapSDK folder (see above) into the Xcode project directory tree: Leap.h and LeapMath.h. When the transfer is completed, a dialog box will appear where you need to specify the method for inserting / linking files to the project; check the box Destination -> Copy items into destination group’s folder (if needed), tick Folders -> Create groups for any added folders and check the project to which files are added below. In addition, we still need a dynamic library. Since the C++ language compiler (LLVM) included with Xcode follows the C++11 standard, you must use a library compiled with its intervention. There is such a lib, it is called (OS X version) libLeap.dylib and is located in the libc++ subdirectory of the lib subfolder of the LeapSDK directory. Liba also needs to be moved to the Xcode system, with the same subsequent passage of the dialog. Now we need to tell the Xcode environment to use the library added to the project. In the project file/directory tree, click on the project name (upper item), the project configuration menu will open. Go to the Build Phases tab. In the upper left corner of the tab, by clicking on the plus sign, from the appeared context menu select New cope files build phase. A collapsed Copy Files panel appears at the bottom of the tab. After expanding it, select Executables from the Destination drop-down list, and in empty list files (below) from the project tree, drag the dynamic lib, with the Copy only when installing check box unchecked. Now she is connected to the project. The next step is for the sensor to transmit "raw" image data of what it sees; in the Leap Motion settings (the Settings item of the context menu of the device icon in the menu bar), on the General tab, check the Allow Images checkbox. The template generated by TinderBox includes several folders, files and necessary frameworks. Because I named the project RawImagesApp, I added the header file RawImages.h. In it I put the connection of the Cinder and Leap header files, the inclusion of the Leap namespace and the declaration of the Leap Motion controller object, in fact, it is the central subject of discussion. In addition, TinderBox generated source for our project, it will serve as a good starting point for development. The cpp file contains the main class (in my case RawImagesApp) of the application, corresponding to the project name and inherited from the Cindera base class - AppNative. The window is created using the CINDER_APP_NATIVE macro. The RawImagesApp class declares and implements the virtual functions of the base class. The setup function is called when the application starts, the code for its initialization is placed here: to display "raw" graphic data in this method, you need to set a special sensor policy flag, for which you need to call the setPolicyFlag method, in which you pass the controller's POLICY_IMAGES value. The update function is called every frame to update; draw is called to redraw the content; mouseDown - when the mouse button is pressed. By default, not all possible functions are enabled, for example, you can add prepareSettings - a function that is called before the window is created and allows you to pass parameters to it. Let's add this function so that when the window is created, it will be larger, and we will also set the refresh rate for it. The declaration inside the RawImagesApp class looks like this:

Void prepareSettings(ci::app::AppBasic::Settings* settings);

and the implementation is like this:

Void RawImagesApp::prepareSettings(Settings* settings) ( settings->setWindowSize(1024, 768); settings->setFrameRate(60.0f); )

I'm sure no comments are needed here. Let's add an OpenGL texture to the main application class: gl::Texture tex; It will be useful to us for the output. In the update function, we will receive images from the sensor frame by frame, then process them and display them on the texture (see source). On each frame, we get a controller frame: Frame frame = controller.frame();. The Frame class object contains all the other objects that the controller generates information about. We just need to extract them from it. By the way, receiving a frame in this way - taking it from the controller on your own (serial polling of the device) - is the simplest and most often used. Any intermediate points are a foregone conclusion: if at the next poll new frame not yet ready, then the old one returns; if several frames are already ready during the subsequent poll, they are placed in the history. There is another way to get frames, but for now we do not need it, and we will transfer its consideration to the next section. Having received a frame, we extract the images taken by the sensor from it: ImageList images = frame.images();. There are two of them, since there are two cameras in the sensor, so there are two images at any given moment. Next, we sequentially process both. First in line: const unsigned char * image_buffer = image.data(); get image data; at a certain point in time, we can receive different images from the controller - not only in content, but also in size. The next line creates a graphical surface object (Surface) included in the Cinder API. Four parameters are passed to its constructor: the width and height of the surface, the use of the alpha channel, the sequence of color channels (the SurfaceChannelOrder::RGBA constant conforms to the standard: red, green, blue, alpha, but there are a number of others, for example, GDI or Quartz use other color sequences). The iterator then iterates over all the pixels of the (so far empty) surface. Inside this loop, the color of the pixels is set. I decided to give the output image a reddish tint (like in DOOM :)). Therefore, the red channel of each pixel is set to a value corresponding to the value from the image data. The rest of the channels are reset. After traversing the entire image, we construct a texture object using the gl::Texture method based on the surface passed in the parameter. If you now display the texture on the screen, it will be too small. Therefore, we scale it first: glScalef(2.0, 3.0, 0.0);. Now let's draw: gl::draw(tex);.

Rice. 5. What the Leap Motion Controller sees

Bones

In the following example, we will display our hands in a machine context, that is, we will draw them in the appropriate coordinates. This task will be more difficult than the previous one, and LeapSDK still provides a rather low-level interface, so we will use ready-made developments to simplify our task. American programmer Stephen Schieberl, nicknamed Ban the Rewind, has developed a couple of classes (Listener inherits from Leap::Listener and Device) that do all the typical work associated with processing and returning device states. In addition, Stephen has placed functions in the file that perform coordinate and matrix calculations, which will allow us to concentrate on higher-level work. First of all, these calculations are due to the fact that, unlike the coordinates of the operating system desktop, where the Y axis grows from top to bottom, the origin for Leap Motion (0, 0, 0) starts in the lower left corner (Y grows from bottom to top), therefore, when using Y-coordinate values, they must be inverted. Additional calculations are performed on vectors and matrices, as described above. So, let's create a new project in the same way as the previous one. Additionally, add the Cinder-LeapMotion.h and Cinder-LeapMotion.cpp files to it (see the materials for the article). In the main application class, the number of member variables has been replenished, the following have been added: mDevice - a reference to the device - an object of a self-written class, mFrame - of the Frame class (we have already considered this class in the previous section), mCamera - an object of the CameraPersp class or Cindera, a method has also been added onFrame (ancestor class callback function), which, by taking a Frame object, makes it current - assigns it to the mFrame member variable. The Setup method enables drawing modes, smoothing lines and polygons; camera initialization: setting the scope (in the constructor parameters), setting the viewpoint (in the lookAt method); then an object of the self-written Device class is created, which includes the three necessary class objects: Controller, Device (from the Leap namespace) and Listener, in addition, a mutex is indispensable. So we come to the second way to receive frames from the device - listening. Our device class is inherited from the Listener class, which allows us to implement this feature, that is, we receive frames from the controller at a frequency corresponding to its operation. When the controller is ready to send a frame, the Listener class calls the onFrame method we overridden and passes the frame (in the parameter) to it, we mentioned this method above. By the way, why do we need a mutex? The point is that when using listener - the callback function - onFrame is called in multi-threaded mode. That is, each of its calls is carried out in an independent thread. Therefore, we need to take care of thread safety at the time of receiving a frame from the device, which is what the mutex serves. When listening, you can also ignore the arrival of a new frame (for example, if the past frame has not yet been processed) and add it to the history (for further processing). Let's return to our code, to the place where the object of our Device class was created. After it is created, a callback function is set for it.

Redrawing

But the most interesting thing happens in the redraw method. First, preparatory actions are performed: clearing the screen, setting the current matrices for the camera, enabling alpha blending, the ability to read and write to the depth buffer, setting the color for drawing. Then direct drawing begins: we receive three-dimensional vectors of the position of the elbow and wrist from the device and use the gl::drawLine method to draw a line between these points. Next, we get the number of fingers and in the loop, using the iterator, we run through the container containing them. In Leap Motion, each finger consists of four parts (phalanges): peripheral, intermediate, proximal and metacarpal. Although the last phalanx is absent on the thumb of a real human hand, it is present here, but has a value of zero. In a nested loop, bypassing all the phalanges, we get the coordinates of their various parts: start, center, end, direction. The coordinates are presented as vectors (Vec3f). Also inside this subcycle, the phalanges are drawn using the drawLine method, to which the found coordinates are passed. Additionally, a container of joints (knuckles) is formed from the first phalanges. When the outer loop exits, lines are drawn connecting the fingers and forming the hands. This is where the redrawing fun ends. Compile and run the program, hold your hands over the sensor, and the outlines of your limbs will be displayed in the window.

Results

Leap Motion is a revolutionary controller that not only replaced the touch screen, but also gave us control over space, making the border between the real world and virtual reality even more transparent. At the software developer level, we get a convenient software interface, which allows you to control all the capabilities of the sensor. Cross-platform developer tools give the latter access to the device in a variety of programming languages, both compiled and interpreted (so far only two of the latter are Python and JavaScript). In addition, the API has a coherent and understandable structure: at each moment of time, the controller takes an image, forms a frame based on it and sends it to top level- into an application program where the programmer, having parsed the frame, works with such entities as hands, fingers, pointers (tools) and more. Due to the device's dual cameras, it is often mounted on virtual reality goggles to create an augmented reality effect, which is achieved due to the presence in the images taken by the cameras of the measured brightness values ​​​​of the infrared emitter, as well as the calibration data needed to correct a complex lens. In today's article, we touched on the topic of creating application solutions that interact with the device through the API. This topic is very extensive, and far from everything was considered - gestures, special movements, touch emulation and much more were left behind. All this and much more, such as using the controller in Windows and the Web, integrating with game / graphics engines, can be a topic for conversation in the coming articles. It all depends on you - write to us, demand to continue :). In the meantime - good luck in all matters and see you on the pages of "Hacker"!

Issue price- 4990 rub.

Contents of delivery- the device itself, 2 USB cords.

Appearance.

When I first picked up a box with a gift to me Leap Motion, a single thought was spinning in his head - what the hell is this all about? Since the dialogue half a year ago with a discussion of this novelty of the electronic market was completely wiped out of my memory, and instructions in Russian were not attached, I had to go to Google for help, which gave me something like the following. :)

In theory. In practice, of course, everything is somewhat different. :)

The device itself looks something like the picture, although the dimensions are slightly larger: 8 cm long and 3 wide. Due to the rubberized lining, it should stand steadily on the table, although personally it always warps a little for me :) It is connected to the computer via a USB-USB 3.0 cable, installed between the monitor and keyboard and works quietly on Windows 7.

Many people write about some special tests when connecting a device, but I didn’t have anything like that, except for registering with Leap Motion Airspace, but more on that below.

Leap Motion - software.

After the system recognized the foreign device and happily announced that it was ready for work, I solemnly waved my hand in front of the monitor, and .. the miracle did not happen. :)

I had to go to the site https://airspace.leapmotion.com/ and download Leap Motion Airspace from there - a program that is a typical distribution service for games and other software created for this device.

Downloaded, installed, registered, installed software, started testing. The first program, Orientation, demonstrates the most spectacular features of Leap Motion. It was rather unusual to see your own upper limbs in this form:

Further on the choice - in the Airspace Store there are games, programs for creating various objects, drawing games, music creators and other equally interesting gizmos. Of course, there are many paid ones, but among the free ones there is something to play with.

Perhaps the most unpleasant discovery here was the separation of software for Mac and Windows, so that for some programs, owners of the "wrong" system will only have to lick their lips.

To do this, go to the section Computer Controls and download there Touchless- for poppy and Windows, respectively.

Leap Motion - set up for normal operation.

Then the most interesting begins. :) Initially, it took me about 5 minutes to even hit the icon on the desktop. It was similar to the feeling that arose during the first use of the computer - everything is scary, it clicks and nothing works. :) So at a certain stage of poking on the screen, bewilderment began to give way to disappointment.

And then came the understanding that it would be nice to get into the settings, if any. Settings was found in the notification area and was a compact icon "Gestures". We right-click on it (oh, what bliss it is when everything works right away and as it should) and select Open the Gestures control panel.

In the opened window Gestures exhibit Accurate pen and touch sensitivity. Then go to the tab Pen Options. Choose a line Double tap Double click and press Options.

Sliders in a new window Speed And Distance move all the way to the side more and test the result in a small window with a door. If everything is done correctly and you have already adapted to the double touch - the door will open. We perform the same procedure on the tab touch.

An unusual gadget jumped into our caddrooffice - Leap Motion, which was introduced last year and made a lot of noise. Its creators are seriously aiming to rethink the management of interfaces, the main goal is to replace the mouse. I’m not sure that this will be the case, nevertheless, it’s more convenient to keep your hands on the table, and constantly waving them you quickly get tired. But some tasks will definitely be more convenient to perform with the help of such a sensor. For example, games, because for all kinds of arcades and timekillers, with which you can relax and forget about work for 20 minutes, Leap Motion is perfect.

How does this whole thing work? In a nutshell, a certain cube is formed in front of the device, with a side of 60 cm, in which the movements of your fingers are recognized with great accuracy. Already, Leap Motion recognizes all 10 fingers, and the accuracy of their movement is captured to a fraction of a millimeter. The delay is also minimal. But lags happen and, so far, quite often. For example, even in the demo application, fingers disappear if they are moved and rotated. And sometimes it seems that the finger comes off. However, the work is already good and in applications like Cut The Rope, where only one finger is involved, lags rarely occur. And here Google Earth I didn’t manage to tame it, the earth rotates in a panic in my hands, and then I immediately find myself under water. And so all the time.

Leap Motion is not just a piece of iron, it is a whole ecosystem. It would be pointless if Leap Motion could only boast of demos. No, the developers have created a hub for computers (Mac and PC) - Airspace. Here you can see everything that has already been downloaded/purchased. There is also a web app store - Airspace Store. Here you need to register, after which you can start shopping. Mostly applications are paid, there are categories, there are sections only for Mac and only for Windows. After purchasing an application or game, you can find the purchase in the hub, from there the download takes place. If you log in on a different computer, all your purchases will be synced.

Leap Motion itself is a compact box, the size of a 128 GB flash drive. The case is made of aluminum, glass and rubberized bottom, which does not allow the device to fidget on the table. Leap Motion is connected to the computer using a USB cable, similar to those for external 2.5″ hard drives. There are 2 such cables in the kit, one meter long, the other half as long. Once the LM is connected to the computer, all that remains to be done is to install the application, which can be downloaded from the website, and register your device.

For now, it's nothing more than a toy. The only thing useful application, which I managed to find - Touchless for Mac, which turns the LM into a trackpad. But again, it is very difficult to use it. I would even say it's unrealistic. So far, Leap Motion is just fun for friends, demonstrating the possibilities with the help of demo applications and games. But, at the same time, it is the first step into the future. I think in two years we will be able to see a similar device in more than one laptop.