Hylynyiv Lyngyrkz

Sensing Glove

input device for real-time sound synthesis and VR, ctm hacklab 2015, HKW Berlin 2016

A first prototype of this sensing controller was built during ctm/cdm hacklab 2015 where it was connected to Max MSP to control real-time sound synthesis.

An augmented version with flex sensors was developed and built by Hylynviy for a theater project of artist Internil and used within the immersive VR experience „Glühende Landschaften“ at the occasion of the Forecast Festival, at HKW, Berlin in 2016.

It was implemented in combination with a 18 sensor full body motion capture system by Optitrack and guaranteed a tracking of every single finger on two hands, which the Optitrack system had not provided.

An atmega328 is hooked up to an accelerometer and gyro and controls a max msp patch. The xyz-coordinates are mapped to various sound synthesis and RGB values.

∆ºª∆º @∂‚@~π∑€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á≠¡}¶∂πø‚π GYRO = new Vector3 (gyro_x,gyro_y,gyro_z); // ACCEL = new Vector3 (
acc_x,acc_y,acc_z);
// MAGN = new Vector3 (mag_x,mag_y,mag_z); // //// //// transform.localPosition = new Vector3 (transform.position.x+1,transform. πø⁄ €π }€{ ∑][«“≠¡}position.y,transform.position.z); // // gameObject.BroadcastMessage ("rotateØ∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒC_CCW", ACCEL); //
// gameObject.BroadcastMessage ("changØ∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒeFreq", mag_y);˜\·∑†∂ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á §C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—©ª∂«•π{| ‚«“≠}¶{≠“ œ«å~‚…∂ •πœ–…∞€¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒ // // gameObject.BroadcastMessage ("changeFreq3", mag_z);‚~∞˜\
·∑†∂ﬁ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒ // // gameObject.BroadcastMessage ("rotation_", ACCEL); // // gameObject.BroadcastMessage ("changeFreq_Sine", mag_y); //// gameObject.BroadcastMessage ("changeGain_Sine", acc_x);«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºª∆º @∂‚@~π∑€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶ // // gameObject.BroadcastMessage ("moveY", y_num); //// ga
meObject.BroadcastMessage ("moveZ", z_num); // }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €πgameObject.BroadcastMessage ("changeFreq2", mag_x); }€{ ∑][€ πø⁄ €π }€{ ∑][«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºœø⁄•∂ºƒç≈©º‚† §C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄
~¢¶¡¿''€ ∫ƒ©ª∂«•π{|«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚º@ø¨©≠}[–…∞º√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ ≠“ œ{≠“ transform.localEulerØ∏Øªº©@√¥≈…å⁄~¢¶¡Angles= new Vector3(-gyro_y, gyro_z, 0.0f);œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∂‚ºº√‚∞ç≈…≈Æ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒ©ª∂«•π{|
‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ gameObject.BroadcastM¡}¶∂πø‚π essage ("rotateC_CW", ACCEL);πø⁄ €π }€{ ∑][ ∆ºª∆º @∂‚@~π∑€ public double increment; public double ºƒç≈©º‚†®ΩΩ⁄π }¶phase; private double sampling_frequency = 48000;ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][€ πø⁄ €π }€{ ∑][«“≠¡}¶∂πø‚π }¶{∆º
œø⁄•∂ºƒç≈ void changeFreq_Sine(int input_acc){⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈ } // void changeGain_Sine(int input_∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ ≠“∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ ≠“acc2){ // gain += input_acc2/10; // }§C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—Ø√¥≈…å⁄~¢¶¡¿''€ ∫ƒ©ª∂«•π{| ‚« }¶{≠“ œ«å~‚…∂ •πœ void OnAudioFilterRead(float[] data, int channels) }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|] increment = Math.Round ( frequen
cy * 2 * Math.PI / ∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓsampling_frequency ,3); for (var i = 0; i < data.Lengt∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓh; i = i + channels) { phase = Math.Round ( phase + increª∂«•π{| ‚«“≠¡}¶ÍŒØ™‚~∞˜\·∑†∂ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á §C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—Ø∏Øment ,3) ; data[i] = (float)(gain*Math.Sin(phas©ª∂«•π{|«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|] œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºª∆º @∂‚@~π∑€ ∫ƒ©ªº©@√¥≈…å⁄~¢¶¡¿'
'€ ∫ƒ©ª∂«•π{|e)); if (channels == 2) data[i + ∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒ ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][«“≠¡}¶∂πø‚π ∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@1] = data[i]; } } ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{∆ºœøåﬂıÓÍŒØ™‚~∞˜\·∑†∂©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{≠frequency = (input_acc+100)*5;“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºœø⁄•∂ºƒç≈©º‚† –…∞€ πø⁄ €π }€{ ∑][if (phase > 2 * Math.PI) phase = 0;“≠¡void changeGain_Sine(int input_∂‚ºº√}¶∂πø‚π } ∆ºª∆º @∂‚@~π∑€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{∆ºœø⁄…∞•π{| ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ §C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶¡¿''€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ ∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞Ï°Í
Æ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶SerialPort sp = new SerialPort("COM3", 9600); // // void Start () { // sp.Open ();∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á // sp.ReadTimeout = 250; //√¥≈…å⁄~¢¶¡¿''€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºœø⁄•∂ºƒç≈©º‚† §C%Y£ﬁ^ÍŒÅﬂ„›Ï°ÍÆ°∏Ø’÷—Ø∏Øªº©@√¥≈…å⁄~¢¶¡ //// session_id += 1; // // } // void Update () { // strArr = null; // // currentI€ πø⁄ €π }€{ ∑][€ πø⁄ €π }€{ ∑][«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ∆ºœø⁄•∂ºƒç≈©
ºªº©@¿''€ ∫ƒ©ª∂«•π{|«“≠¡}¶∂πø‚π }¶{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄|]“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][ ∆ºª∆º @∂‚@~π∑€ ∫ƒ©ª∂«nput = sp.ReadLine(); // strArr = currentInput.Split (","•∂ºƒç≈©º‚†®ΩΩ⁄|]∂‚ºº√‚∞ç≈…≈åﬂıÓÍŒØ™‚~∞˜\·∑†∂ﬁ^·™ÍÁˇ™©ª‚ﬂÍ–∂@ç∆µ∞≈µ~Í∏™ÍÅ°∏Á≠¡}¶∂πø‚π }¶
{∆ºœø⁄•∂ºƒç≈©º‚†®ΩΩ⁄π }¶{≠“ œ«å~‚…∂ •πœ– [0]); // // // // s1 = strArr[0]; // s2 = strArr[1];¡¿''€ ∫ƒ©ª∂«•π{| ‚«“≠¡}¶∂πø‚π }¶{≠“ œ«å~‚…∂ •πœ–…∞€ πø⁄ €π }€{ ∑][print (currentInput);‚†®ΩΩ⁄|]°∏Ø’÷—Ø∏Ø