【This Web Policy】
You should make JavaScript available to read this page.
Please enable the execution of JavaScript!












































【How to use Kinect Sensor on MikuMikuDance Ver.7.39】
　 　 【Link Free】　Private Laboratory Neo-Tech-Lab.co.uk
【Reporter】 ☆Tomoaki Ueda☆ 		Published Date May 24, 2011
Welcome to neo-tech-lab.jp.



【Google Chrome is the best browser】

●Let's download the latest version of Chrome here.

【News】

Now, no article on this issue.
【Photo 1】Motion Capture Sensor for Microsoft's XBOX360, Kinect Sensor

【Hello! Welcome to Kinect World!】

   Do you know the Kinect Sensor? The Kinect Sensor is the motion capture sensor for Microsoft's XBOX360. It is shown as Photo 1. We can buy only the Kinect Sensor. The price is about USD$150. You can enjoy the Kinect world on your windows machine without XBOX360. It has a USB interface. You can connect Kinect to your windows machine by using the freeware OpenNI.
　　　　　

【日本語版はこちら】 ⇒ OpenNI導入法とサンプルプログラム

【The feature of MikuMikuDance Ver.7.xx】

   Yu Higuchi who is the author of MikuMikuDance had developped MikuMikuDance Ver.7.xx which enables you to teach your favorite model your wonderful motion by using Kinect.
   At first, please check Higuchi's demonstration movies.
      ...................................   Waaaaaaooooooooouuuuuuuu! Great! Wonderful! "＼(^o^)／"
   But.....
   Keep your head! To enjoy Kinect world on MikuMikuDance, you should do the difficult preparation.

[Movie 1] MikuMikuDance with OpenNI (Kinect) test
(Author: Yu Higuchi)
[Movie 2] MikuMikuDance with OpenNI (Kinect) test 2
(Author: Yu Higuchi)

[Movie 3] MikuMikuDance with OpenNI (Kinect) test 3
(Author: Yu Higuchi)
[Movie 4] I also had tested MikuMikuDance Ver.7.00 on my computer. What an easy teaching process! Superb!

【The best method by using Microsoft Kinect for Windows SDK Version1】(2012/02/01)

　　　　　　　　　　　　　　　for only Windows7
In order to enjoy MikuMikuDance with Kinect, we have two choices now.
The first method is to use Microsoft Kinect for Windows SDK Version1 and MoggNUI.
If your computer is Windows7, I recommend this method. If you don't have Windows7, you should select another method.

1. Download and Install Kinect for Windows SDK Version 1.0.
　　　http://www.microsoft.com/en-us/kinectforwindows/
2. Please execute the sample program named as Skeletal Viewer of Kinect for Windows SDK.
　Kinect for Windows SDK Version 1.0 has this sample program.
Is it working? Please confirm it.
3. If you will see the error message regarding MSVCR100.dll, you should install the following package and DirectX9 runtime.
　3.1 Microsoft Visual C++ 2010 Redistributable Package (x86)
　3.2 MicrosoftのDirectX End User Runtime
4. Download MikuMikuDance_v739dot.zip and unzip it.
　　　http://www.geocities.jp/higuchuu4/pict/MikuMikuDanceE_v739dot.zip
5. Download MoggNUI and unzip it.
　　　https://sites.google.com/site/moggproject/enghome
6. Copy the following files to MikuMikuDance 'Data' folder.
　　　　- DxOpenNI.dll
　　　　- MoggNuiConfig.exe
7. Activate MoggNUI with MoggNuiConfig.exe. You may use the default settings.

【１】Microsoft Kinect for Windows SDK Version1 Download Page

⇒⇒

【４】MikuMikuDance(DirectX9 Ver. = Version 7.39.dot)

【４】MikuMikuDance(64bitOS Ver. = Version 7.39.dot)


【５】Mogg's NUI Version 0.3.2

The second method by using OpenNI

【1st Stage: Let's set up OpenNI on your PC to use Kinect Sensor on your OS】

【About OpenNI】

Please check this page about OpenNI. I don't konw the details. But I think that this software had been developed by PrimeSense Ltd. which is the company in Israel.

【Index】

●【Step_1: Download】
●【Step_2: Install Device Driver】
●【Step_3: Install OpenNI】
●【Step_4: Install NITE】
●【Step_5: Install Sensor】
●【Step_6: Modify the environmental parameter】
●【Step_7: Displace xml-file for OpenNI】
●【Step_8: Displace xml-files for NITE】
●【Step_9: Operation Check】

【Step_1: Download】

Please download 3 files as followings.
1-1．Please download the Device Drivers for Kinect (PrimeSensor Modules for OpenNI) from here (github Avin2). After clicking the link named as '8f199ca', you need wait that the downloader will be ready. Just seconds, please. As the status of the downloader is frequently busy, you need usually wait over 30 seconds. If you move the mouse cursor onto the 'Downloads' button, the 'Downloads' button will become blue after the downloader is ready.
　　avin2 pushed to master at avin2/SensorKinect April 16, 2011　[Tree: 8f199ca]
　　Click the button named as 『Downloads』. And then click the button named as 『Download.zip』 to download zip file named as 『avin2-SensorKinect-8f199ca.zip』. The file size is 11.2 mega bytes.





1-2．Please download the latest unstable version of 『OpenNI Binaries』 named as 『OpenNI Unstable Build for Windows x86 (32-bit) v1.1.0.41 Development Edition』 from OpneNI.org.
You can get the installer named as 『OpenNI-Win32-1.1.0.41-Dev.msi』. The uploaded date could be April 18, 2011. The file size is 14 mega bytes.
Even if you have Windows7 type 64bits, you should download 『OpenNI-Win32-1.1.0.41-Dev.msi』.



1-3．Please download the latest unstable version named as 『PrimeSense NITE Unstable Build for Windows x86 (32-bit) v1.3.1.5 Development Edition』 from OpneNI.org.
This is the installer of 『OpenNI Compliant Middleware Binaries』 named as 『NITE-Win32-1.3.1.5-Dev.msi』. The uploaded date could be April 18, 2011. The file size is 37 mega bytes.
Even if you have Windows7 type 64bits, you should download 『NITE-Win32-1.3.1.5-Dev.msi』.



1-4. Now you could get 3 files. And you should unzip 『avin2-SensorKinect-8f199ca.zip』. After unzip it, you would find the folder named as 『avin2-SensorKinect-8f199ca』 on your desktop.

Q1: Why do we download the Development Edition?
A1: The Development Edition has the binaries of the various sample programs. We can use them for checking operation.

Q2: I have Windows7 64bits.
A2: Even if you have Windows7 64bits, you should download 32bits OpenNI and 32bits NITE.
If you would not do so, you could not get success because the avin2's installer has a bug now.

【Step_2: Install Device Driver】

2-1．Supply the electric power to Kinect Sensor. Connect the USB cable to your PC's USB port at Kinect Sensor power ON. In the case that your PC is Windows7 machine, the process for installing the device driver of Kinect Sensor will be failed perfectly. Calm down! It is normal absolutely.

2-2．Confirm the status of your computer by do 『Computer』⇒『System Property』⇒『Device Manager』. You will find 『Xbox NUI Motor』 in the other devie.

2-3．In the case that your Windows7 would be 64bits type OS, please look for the following execute-file in the folder created by unzipping 『avin2-SensorKinect-8f199ca.zip』. Check 『avin2-SensorKinect-8f199ca/Platform/Win32/Driver』. You can find 『dpinst-amd64.exe』 in this folder. Please execute it. This program (dpinst-amd64.exe) enables Windows7 the auto-detection for Kinect Sensorset.
In the case of Windows7 32bits type OS or Vista, you need execute the program named as 『dpinst-x86.exe』.

2-4．After execution, your Windows detect the Kinect Sensor and install the three device drivers (Kinect Audio, Kinect Camera, Kinect Motor).
　　(The previous version didn't have 'Kinect Audio'.)





■Sorry. I don't have English mode windows.

【Step_3: Install OpenNI】

3-1．Install OpenNI by using the installer named as 『OpenNI-Win32-1.1.0.41-Dev.msi』. You can complete the installing process with the default setting. In the case that your OS is 64 bit, please execute this version for 32bis OS. I could not success the install if I use the version for 64bits OS. (If you use the 64bits version, you will see the error message at the step 5. It is just the bug of the program for installing 『PrimeSensor』.)
If your OS is 64 bits, the folder named as 『OpenNI』 will be created in the folder of 『C:\Program Files (x86)』.
If your OS is 32 bits, the folder named as 『OpenNI』 will be created in the folder of 『C:\Program Files』.

【Step_4: Install NITE】

4-1．Install NITE of PrimeSense Ltd. by using the installer named as 『NITE-Win32-1.3.1.5-Dev.msi』 with the default settings. This is also the 32 bits type.
License Key is 『0KOIk2JeIBYClPWVnMoRKn5cdY4=』. (It is written on the previous page for downloading the NITE installer.)
If your OS is 64 bits, the folder named as 『PrimeSense』 will be created under 『C:\Program Files (x86)』. And the folder of 『NITE』 is created there.

【Step_5: Install Sensor module】

5-1．Install PrimeSensor module to use OpenNI.
Use the installer named as 『SensorKinect-Win-OpenSource32-5.0.1.msi』 under 『avin2-SensorKinect-8f199ca/Bin』 folder.
The 『Sensor』 folder will be created under 『C:\Program Files (x86)/PrimeSense』 in the case that your OS is 64 bits.

【Step_6: Modify the environmental parameter】

6-1．Perhaps it may be the bug of the installer. One of the environmental parameter, OPEN_NI_BIN should be 『Bin』。But it would be 『bin』. So modify it.
Move from 『System and Security』 to 『System』. Or 『Computer』⇒『Property of System』. Please click 『System Detail property』. You can find the button named as 『Environmental Prameter (N)』. Click it. Look for 『OPEN_NI_BIN』. Select it and click the button named as 『Edit(I)』. Then modify it. And Click the OK button.

【Step_7: Replace an xml-file for OpenNI】

7-1．Create the text file named as 『SamplesConfig.xml』 as shown the following under Desktop by using by notepad.exe. And copy (or overwrite) this file into the folder named as 『C:\Program Files (x86)\OpenNI\Data』. You can't overwrite it after modifying the existing file under the 『C:\Program Files (x86)\OpenNI\Data』 folder directly. It is the security limitaion.
By the way, my PC is laptop type. So the screen size is small. I had written the resolution items as 『xRes="640" yRes="480" FPS="30"』. However if your PC is the desktop type, you can wrote 『xRes="1280" yRes="1024" FPS="15"』, I think. (But I didn't check it.)

『SamplesConfig.xml』

<OpenNI>
	<Licenses>
		<License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>
	</Licenses>
	<Log writeToConsole="false" writeToFile="false">
		<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
		<LogLevel value="3"/>
		<Masks>
			<Mask name="ALL" on="true"/>
		</Masks>
		<Dumps>
		</Dumps>
	</Log>
	<ProductionNodes>	
		<!-- Normal Image -->
		<Node type="Image" name="Image1" stopOnError="false"> 
			<Configuration>
				<Mirror on="true"/>
			</Configuration>
		</Node>	
	
		<!-- HighRes Image -->
		<!--
		<Node type="Image" name="Image1" stopOnError="false"> 
			<Configuration>
				<MapOutputMode xRes="640" yRes="480" FPS="30"/>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		-->
		
		<!-- Normal IR -->
		<!--
		<Node type="IR" name="IR1">
			<Configuration>
				<MapOutputMode xRes="640" yRes="480" FPS="30"/>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		-->
		
		<!-- HighRes IR -->
		<!--
		<Node type="IR" name="IR1">
			<Configuration>
				<MapOutputMode xRes="640" yRes="480" FPS="30"/>
				<Mirror on="true"/>
			</Configuration>
		</Node>		
		-->
		
		<Node type="Depth" name="Depth1">
			<Configuration>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<!--
		<Node type="Audio" name="Audio1">
		</Node>
		-->
	</ProductionNodes>
</OpenNI>

【Step_8: Replace 3 xml-files for NITE】

8-1．As shown in the above section, create 3 text files on your Desktop by using notepad.exe. And copy Sample-Scene.xml, Sample-Tracking.xml and Sample-User.xml into 『C:\Program Files (x86)\PrimeSense\NITE\Data』 folder.

『Sample-Scene.xml』

<OpenNI>
	<Licenses>
		<License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>
	</Licenses>
	<Log writeToConsole="true" writeToFile="false">
		<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
		<LogLevel value="3"/>
		<Masks>
			<Mask name="ALL" on="false"/>
		</Masks>
		<Dumps>
		</Dumps>
	</Log>
	<ProductionNodes>
		<Node type="Depth">
			<Configuration>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<Node type="Scene" />
	</ProductionNodes>
</OpenNI>

『Sample-Tracking.xml』

<OpenNI>
	<Licenses>
		<License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>
	</Licenses>
	<Log writeToConsole="true" writeToFile="false">
		<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
		<LogLevel value="3"/>
		<Masks>
			<Mask name="ALL" on="false"/>
		</Masks>
		<Dumps>
		</Dumps>
	</Log>
	<ProductionNodes>
		<Node type="Depth">
			<Configuration>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<Node type="Gesture" />
		<Node type="Hands" />
	</ProductionNodes>
</OpenNI>

『Sample-User.xml』

<OpenNI>
	<Licenses>
		<License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>
	</Licenses>
	<Log writeToConsole="true" writeToFile="false">
		<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
		<LogLevel value="3"/>
		<Masks>
			<Mask name="ALL" on="false"/>
		</Masks>
		<Dumps>
		</Dumps>
	</Log>
	<ProductionNodes>
		<Node type="Depth">
			<Configuration>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<Node type="User" />
	</ProductionNodes>
</OpenNI>

【Step_9: Operation Check】

Are you ready?
You can execute the binaries of the Kinect Sample program.
For example, execute 『NIUserTracker.exe』 in the 『c:\Program Files (x86)\OpenNI\Samples\Bin\Release\』 folder.
After running and user recognition, you can see the skelton by the fist pump.

【2nd Stage: Download MikuMikuDance Ver.7.xx】

【About MikuMikuDance Ver.7.xx】

1-1) Download 『MikuMikuDance Ver.7.xx』 and 『DxOpenNI.zip』 on Higuchi's download site.



1-2) Unzip the zip-file named as 『DxOpenNI.zip』. You can see DxOpenNI.dll and SamplesConfig.xml in the 『DxOpenNI』 folder. Copy them into the 『Data』 folder of MikuMikuDance.

1-3) Replace 『SamplesConfig.xml』 file. 
<OpenNI>
	<Licenses>
		<License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>
	</Licenses>
	<Log writeToConsole="false" writeToFile="false">
		<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->
		<LogLevel value="3"/>
		<Masks>
			<Mask name="ALL" on="true"/>
		</Masks>
		<Dumps>
		</Dumps>
	</Log>
	<ProductionNodes>
		<Node type="Image" name="Image1">
			<Configuration>
				<MapOutputMode xRes="640" yRes="480" FPS="30"/>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<Node type="Depth" name="Depth1">
			<Configuration>
				<MapOutputMode xRes="640" yRes="480" FPS="30"/>
				<Mirror on="true"/>
			</Configuration>
		</Node>
		<!--
		<Node type="Audio" name="Audio1">
		</Node>
		-->
	</ProductionNodes>
</OpenNI>

【3rd Stage: Let's enjoy MikuMikuDance Ver.7.xx with Kinect】
      Now look at Higuchi's movie again. And let's enjoy MikuMikudance with Kinect Sensor.

【Step3-1: After loading Model, Set physical engine】


【Step3-2: Set Kinect Mode】

3-2) motion capture(K) - Kinect(K)
　After you started the Kinect mode, you woukd see the red man on the screen.
　Ok? Move your body!



【Step3-3: Start Kinect Capture】

3-3) motion capture(K) - capture(C)
　You can start or stop by using this menu item.













【注意】　以下のコードは、2011年4月版のOpenNI.Net.dllを使った場合のコードです

2011年1月のC# Wapperから見て、かなり構造的な変更が何か所もあったようです。
どうやらプロパティーでもメソッドでもなく、Public変数になっている部分とか、型の変更とか...
一から調べなおすのは面倒だったので、コンパイルしてエラー箇所を機嫌が良くなるように修正するという後ろ向きな修正方法で、以前に作成したサンプルプログラムを動くように修正しました。
取り敢えず、handsGeneratorはまだ復旧できていませんが、RGBカメラとかDepthバッファとかは以前のように動かすことができました。handsGenerator関連は外してもらっても問題ありません。
下にサンプルプログラムのソースを掲載しておきます。

【サンプルプログラムのプロジェクトファイル一式(zip形式) 】

【Kinect Sensorの使い方】

　C# WapperのOpenNI.net.dllを使えば、Visual C#でKinect Sensorを使うことができます。以下に最もポピュラーなRGBカメラの読出/表示、デプスバッファの読出/表示、ユーザー切り出し、骨格情報の読出しのサンプルプログラムを示します。
　スクリーンショットのデプスバッファのカラー表示画面で、最も手前にあるはずの左手の横に白い部分がありますが、これはデプスバッファの数値として0が返されてくる部分なのです。対象がカメラに近いほどこの白い領域がずれて目立ちます。Prime Sense社の３次元座標検出方法に強く関係していると考えられますが、詳細は不明です。
　３次元座標の取得処理時には赤色レーザーが発光しますので、もしかするとレーザー光線をスキャンして２つのカメラを使った視差を求める方法かもしれません。この場合、片側のカメラに光点がなければ３次元座標(深さｚ値)を求めることができないので、値が0になるのかもしれません。


【図xx】RGBカメラ、デプスバッファのカラー表示、ユーザー切り出し、骨格情報読出し表示の例

【図xx】RGBカメラ、デプスバッファのモノクロ表示、ユーザー切り出し、骨格情報読出し表示の例

【図xx】近くほど白い領域がずれて目立つ。Prime Sense社の３次元座標検出方法に関係していると考えられる。

【Kinectセンサについて】

2011/04/03 追記

■Microsoft社XBOX用モーションキャプチャーセンサKinect

■Visual C#でKinectアプリケーション開発が行えます。 		【注】Kinect(キネクト)センサは、Microsoft社のXBOX360用のモーションキャプチャーセンサ(1万5千円弱)で単独で購入することができます。640×480画素のRGBカメラのほか、赤外線カメラも搭載しており、毎秒30フレームの速度で深さ情報(depth buffer)を得る事ができます。USBインターフェースなのでWindowsパソコンに接続可能です。2011年1月6日にC#で使える暫定版OpenNI.net.dllがオープンソースで公開されています。OpenNI.net.dllを使えば上の写真のように基本的な骨格情報(関節の３次元座標)をリアルタイムに得られます。下の動画は磁界・超音波センサに応用してみた事例です。トランジスタ技術2011年2月号の拡張現実センサの進化形に該当します。(詳細は別ページで記載の予定。現在、導入メモ等と一部ソースを記載。[2011/02/05 - 2011/03/31])