Sound of Vision: natural sense of vision through acoustics and haptics

(Co-principal investigator from the Lodz University of Technology project participant in the Horizon 2020 project ID No. 643636 carried out in years 2015–2018.)

Sound of Vision (Natural sense of vision through acoustics and haptics) will design, implement and validate an original non-invasive, wearable hardware and software system to assist visually impaired people by creating and conveying an auditory representation of the surrounding environment. This representation will be created, updated and delivered to the blind users continuously and in real time. In addition to the auditory representation, haptics will be used moderately as an additional channel to convey some of the most relevant information. The system will help visually impaired people to both perceive and navigate in any kind of environment (indoor/outdoor), without the need for predefined tags/sensors located in the surroundings and regardless of the lighting conditions.

(Co-principal investigator in the National Centre for Research and Development grant no. NR02-0083-10 carried out in the Institute of Electronics, Technical University of Lodz in years 2010–2013.)

The aim of this project is to develop a prototype system enabling the blind and visually impaired independent travel and access to urban infrastructure. The envisioned system features a distributed architecture. It comprises wearable units (dedicated and smartphones) communicating wirelessly with stationary beacons and wide area networks. The visually impaired user will decide of system configuration that best aids his disability.

(Co-principal investigator in the State Committee for Scientific  Research grant, 2007–2010)

The aim of the project is to develop a mobile prototype navigational system for the blind. The system is comprised of the following interconnected modules: electronic travel aid (ETA) for obstacle avoidance and environment perception (device model currently worked on in the Institute of Electronics), satellite global navigation system (GPS or Galileo), digital maps of urban terrain (available from the City of Lodz Land Information System). An innovative concept of the envisioned system is the fusion of data obtained from: inertia sensors, stereoscopic images, electronic compass, and satellite navigation system for a sustained identification of local and global geographic position of a blind user in the urban environment.

(Principal investigator in the State Committee for Scientific  Research grant, 2004–2007)

The project is aimed at construction of a prototype portable system aiding the blind in space perception. Fast methods for 3D scene reconstruction from a pair of stereo images will be worked out. The reconstructed scene will be converted into a virtual acoustic representation of space. Video and acoustic signal processing algorithms will be implemented on DSP platforms for real time operation. The system will be capable of communicating with portable electronic devices like: GSM telephone, GPS receiver and Braille printer.

Application of computational intelligence methods for fault detection in electronic circuits (Principal investigator in the State Committee for Scientific  Research grant, 2004–2005)

Real-time generator of analog ECG signals from MIT/BIH Arrhythmia Database digital records (Co-principal investigator in the R&D contract with the Institute of Medical Systems in Zabrze, Poland, 2000)

Air pollution monitoring by means of artificial neural networks (Co- principal investigator in the State Committee for Scientific Research grant no. 3T09 024 13, 1997–2000)

Artificial neural network for dynamic system parameter identification: theoretical development, design and electronic implementation, (Co-principal investigator in the State Committee for Scientific Research grant no. 8T11 F01010, 1995-1998)

A microcomputers system for analysis of X-ray images of wheat grains (Principal investigator in the  R&D contract with the Institute of Agrophysics in Lublin, Polish Academy of Sciences, 1995–1996)

Modelling of biomaterials drying processes by means of artificial neural networks (Co-principal investigator in the State Committee for Scientific Research grant no 3P40502807, 1994–1997)

Neurodynamic modelling of the human heartbeat (PhD research project sponsored in part by the European Community COPERNICUS  Programme, 1990-1992)

A microcomputer system for processing and analysis of medical images using IBM PC computers (Member of team in the R&D contract granted by the Polish National Research Program CPBR 8.6, 1988-1990)

BOOKS

Strumillo P. "Personal navigation systems for the blind and visually impaired", Lodz University of Technology, Lodz 2012 (in Polish)

Strumillo P. (Ed.) "Advances in sound localization", InTech, Croatia, 2011

Kaminski W., Strumillo P., Tomaczak E. "Application of soft computing methods for air pollution modelling", Polish Academy of Sciences Publishing House, Lodz, 2005 (in Polish)

JOURNALS AND BOOK CHAPTERS

Kopczyński B., Strumiłło P., Niebudek-Bogusz E. "Assessment of vocal folds phonation by means of computer analysis of laryngovideostroboscopic images – a pilot study (in Polish)",Otorynolaryngologia 2014, 13(3),: 147-154

Niebudek-Bogusz E., Strumiłło P., Wiktorowicz J., Śliwińska-Kowalska M., "Comparison of cepstral coefficients to other voice evaluation parameters in patients with occupational dysphonia (in Polish)", Medycyna Pracy 2013;64(6):805–816

Niebudek-Bogusz E., Grygiel J., Strumiłło P., Śliwińska-Kowalska M., "Nonlinear acoustic analysis in the evaluation of occupational voice disorders (in Polish)", Medycyna Pracy, 2013, vol. 64, nr 1, str.29-35.

Bujacz M., Skulimowski P., Strumillo P., "Naviton — A Prototype Mobility Aid for Auditory Presentation of Three-Dimensional Scenes to the Visually Impaired", Journal of the Audio Engineering Society, vol. 60, no. 9, 2012 September, pp. 696-708.

Barański P., Strumiłło P., "Enhancing positioning accuracy in urban terrain by fusing data from a GPS receiver, inertial sensors, stereo-camera and digital maps for pedestrian navigation", Sensors, 2012, vol. 12, no. 6, pp. 6764-6801.

Kudrynski K., Strumillo P., Ruta R., "Computer Software Aiding Studies of HRV, TWA and HRT Analysis in ECGs", Medical Science Monitor, vol. 17, no. 9, 2011, pp. 63-71.

Krolak A., Strumillo P. "Eye-blink detection system for human-computer interaction", International Journal on Universal Access in the Information Society, vol. 10, 2011, DOI 10.1007/s10209-011-0256-6.

Strumillo P. "Electronic systems aiding spatial orientation and mobility of the visually impaired",In: Z.S. Hippe, J.L. Kulikowski, Y. Mroczek (Eds.) Human-Computer Systems Interaction. Backgrounds and Applications 2, Springer-Verlag Co., Advances in Soft Computing, 2011, pp. 373-386.

Baranski P., Polanczyk M., Strumillo P. "Fusion of data from inertial sensors, raster maps and GPS for estimation of pedestrian geographic location in urban terrain", Metrology and Measurement Systems, vol. 18, no.1, 2011, pp. 145-158.

Dobrucki A., Plaskota P., Pruchnicki P., Pec M., Bujacz M. and Strumillo P. "Measurement system for personalized head-related transfer functions and its verification by virtual source localization trials with visually impaired and sighted individuals", Journal Audio Engineering Society, vol. 58, no. 9, September 2010, pp. 724-738.

Strumillo P., Skulimowski P., Polańczyk M., "Programming Symbian Smartphones for the Blind and Visually Impaired", in E. Kącki, M. Rudnicki, J. Stempczyńska (Eds.): Computers in Medical Activity, Advances in Intelligent and Soft Computing, Springer 2009, pp. 129-136.

Skulimowski P., Strumillo P., "Refinement of depth from stereo camera ego-motion parameters", Electronics Letters, vol. 44, no. 12, 2008, pp. 729-730.

Strumillo P., Materka A., "Electronic systems aiding the blind and the elderly (in Polish)", Elektronizacja, nr 11, pp. 28-31, 2007.

Strumillo P., Strzelecki M., "Application of coupled neural oscillators for image texture segmentation and biological rhythms modelling", International Journal of Applied Mathematics and Computer Science, vol.16, No.4., 2006, pp. 101-111

Koszlaga J., Strumiłło P., „Discovery of linguistic rules by means of RBF network for fault detection in electronic circuits”, in Lecture Notes in Artificial Intelligence (Eds. L. Rutkowski, J. Siekman, R. Tadeusiewicz, L. Zadeh), Springer Verlag, Proceedings of 7-th International Conference on Artificial Intelligence and Soft Computing – ICAISC 2004, Zakopane 2004, pp. 223–228.

Strumillo P., Ruta J., "Poincare mapping for detecting abnormal dynamics of cardiac repolarization", IEEE Engineering in Medicine and Biology Magazine, vol. 21, no. 1, pp. 62–65, 2002.

Strumillo P., “Nested median filtering for detecting T-wave offset in ECGs”, Electronics Letters, 4^th July 2002, vol. 38, no. 14, pp. 682–683.

Strumillo P., “Modelling and analysis of electrocardiograms by means of nonlinear transformations and circuits”, (in Polish), DSc dissertation, Technical University of Lodz, 2002.

Ruta J., Strumillo P., “Usefulness of the Poincare maps in detection of T–wave alternans in precordial leads of standard ECG – a comparison with the spectral method”, Medical Science Monitor, 2001, vol. 7, no. 3, pp. 471–476.

Strumillo P., W. Kaminski, “Orthogonalisation procedure for training radial basis functions neural networks”, Bulletin of the Polish Academy of Sciences, 2001, vol. 49, no. 3,  pp. 479–492.

Strumillo P., “2-D spatiotemporal imaging of T-wave alternans in precordial ECG leads by means of Poincare sections”, Journal of Medical Informatics and Technologies, vol. 2, 2001, pp. MT67–MT73.

Strumillo P., "Matching pursuit of basis in two-layer neural networks for signal approximation,'' International Journal of Computers, Systems and Signals, vol. 2, no. 1, pp. 43–53, 2001.

Niewczas J., Kudra T., Strumillo P., W. Wozniak, “Stress cracking as a measure of grain quality after thermal drying”, Developments in drying, vol. II, (A. S. Mujumdar, Ed.), Chapter one, pp. 1–35, Kasetsart University Press, Bangkok, Thailand, 2000.

Strumillo P., Kaminski W., “Modelling and Interpolation of Seasonal Air Temparature Changes in Central Europe”, in Computational Intelligence and Applications, P. Szczepaniak (Ed.), Springer Verlag, 1999, pp. 302–310.

Kaminski W., Strumillo P., Tomczak E., Neurocomputing approaches to modelling of drying process dynamics, Drying Technology, 16(6), pp. 967–992, 1998.

Strumillo P., Niewczas J., Szczypinski P., Makowski P., Wozniak W., Computer system for analysis of x–ray images of wheat grains,Int. Agrophysiscs, 1999, vol. 13, No. 1, pp. 133–140.

Kaminski W., Strumillo P., "Kernel Orthonormalization in Radial Basis Function Neural Networks", IEEE Transactions on Neural Networks, vol. 8, no. 5, pp. 1177– 1183, 1997.

CONFERENCES (recent):

Bujacz M., Skulimowski P., Strumillo P. "Sonification of 3D scenes using personalized spatial audio to aid visually impaired persons", The 17th International Conference on Auditory Display (ICAD-2011) June 20-24, 2011, Budapest, Hungary, (Conference CD).

Baranski P., Polanczyk M., Strumillo P. "A remote guidance system for the blind", Proceedings of the 12th International Conference on e-Health Networking, Application & Services (Healthcom2010), 1-3 July 2010, Lyon, France, pp. 386-390.

Kudrynski K., Strumillo P. "Real-Time Estimation of Heart Rate Variability Parameters from Passband Filtered Interbeat Interval Series",Computing in Cardiology, Hangzhou, China, 2011, vol. 38, pp. 297-300.

Grygiel J., Strumillo P., Niebudek-Bogusz E., "Application of mel cepstral processing and support vector machines for diagnosing vocal disorders from voice recordings",Proceedings of the IEEE Signal Processing Conference, SPA2011, 29-30 September 2011, pp. 149-152.

Strumillo P.,"Electronic Interfaces Aiding the Visually Impaired in Environmental Access, Mobility and Navigation", 2010 IEEE Conference on Human System Interaction, 13-15 May, Rzeszow, Poland, pp. 17-24.

M. Bujacz, P. Barański, M. Morański, Strumillo P., A. Materka "“Remote mobility and navigation aid for the visually disabled”", Proc. 7th Intl Conf. on Disability, Virtual Reality and Assoc. Technologies with Art ArtAbilitation, in P.M. Sharkey, P. Lopes-dos-Santos, P.L. Weiss & A. L. Brooks (Eds.), pp. 263–270, Maia, Portugal, 8–11 Sept. 2008.

P. Skulimowski, Strumillo P., ” Obstacle localization in 3D scenes from stereoscopic sequences”, Proceedings of the 15th European Signal Processing Conference (EUSIPCO 2007), September 3-7, Poznań, Poland, pp. 2095-2099

M. Pec, M. Bujacz, Strumillo P., ” Personalized head related transfer function measurement and verification through sound localization resolution”, Proceedings of the 15th European Signal Processing Conference (EUSIPCO 2007), September 3-7, Poznań, Poland, pp. 2326-2330.

D. Rzeszotarski, P. Skulimowski, Strumillo P.” A method for verification of dense disparity maps computed from the matching algorithm implemented in the stereovision system”, Signal Processing Symposium SPS 2007, Jachranka Village May 24-26, 2007.

A. Kozłowski, Strumillo P.” Text localization in natural scene images using multiple classifiers”, Signal Processing Symposium SPS 2007, Jachranka Village May 24-26, 2007.

Rudnicki M., Strumillo P.” A Real-Time Adaptive Wavelet Transform-Based QRS Complex Detector”, Adaptive and Natural Computing Algorithms, 8th International Conference on Adaptive and Natural Computing Algorithms, Warsaw, Poland, April 2007, Lecture Notes in Computer Science (LNCS) Springer-Verlag, vol. 2, pp. 281-289.