Mobile system to assist the teaching of geography for persons with visual impairments

Author: Dagna Turant

The purpose of this engineering thesis was to build a flexible system providing sound support for hypsometric maps. The system consists of two applications. The desktop application assists the sighted user at preparing the map. It helps creating map-related data like map legend and location of cities and rivers and preprocesses the image. The mobile application targets persons with visual impairments. The application represents the graphical map data in the audio form using both sound of different frequencies to communicate the elevation change and speech synthesis to announce cities and rivers. The user can also learn about the distance between two given cities which is communicated by the synthesizer after the cities have been tapped. Another feature is zoom, which enables enlarging a given area of the map. Application at the zoomed state provides the user with more data about a given area. The mobile application was tested by blindfolded users and received positive feedback.
The desktop application was written using JavaFx technology. The mobile application was written in Java for the Android platform.

More info:

Skulimowski Piotr, Turant Dagna; “Mobilny system wspomagający naukę geografii dla osób z niepełnosprawnością wzroku, s.418-423”, XVI Krajowa Konferencja Elektroniki KKE 2017.Darłowo, 2017, Przegląd Elektrotechniczny, 2017, rocznik 2017, nr 8, s. 124-126; ISSN: 0033-2097.

Object recognition in 3D scenes with the use of the SIFT algorithm dedicated for mobile platforms

Author: Karol Matusiak (2012)

The aim of the master’s thesis was to implement an application for mobile devices which would enable identification of everyday life objects in images recorded by the built-in camera. Analysis of the image identification methods demanding relatively low computing complexity was performed and the requirements of the designed application were determined. On the basis of the analysis it was decided to implement two versions of application: one using standard Scale-Invariant Feature Transform and a second one linking the SIFT transform and the Features from Accelerated Segment Test algorithm used for keypoint detection. Both algorithms provide the detection of stable characteristic features of images. The SIFT transform builds descriptors which are to a considerable degree independent from the conditions of image recording, such as: rotation, noise, scale and brightness changes. As the target mobile platform, a group of developed mobile phones with operating system Android was selected.
The program uses the touch screen interface of the mobile device. The recording of an image is done through the digital camera built into the device. The program enables the user to build a database of patterns and to compare the recorded images with these patterns using a modified nearest neighbor classifier.
Application tests were conducted on 4 objects, whose patterns were created and stored in the database. Afterwards, 12 test photos were compared against the database contents. The test results have shown that the performance of the application using the FAST algorithm is 3,5 times faster than the version using standard SIFT. In case of proper detection both programs had same success rate – 83%, but with more accuracy for the standard SIFT. In the summary, the application performance assessment and the possibilities of its improvement and development are proposed.
The application can be used as an aid for people with sight disabilities, city or tourist guides or in virtual reality systems.

Mobile system aiding visually impaired in mathematics learning

Author: Izabela Borowiecka

The aim of the thesis was to develop a method to sound graphs of linear function and its implementation in mobile system aiding mathematics learning. The application is for visually impaired and blind people, therefore all available options are presented by voice communications. The major feature is to express graph of linear function, which coefficient are selected by user, with sound. Moving finger along the graph, generate high frequency tone and activate vibrations, as it moves away, the frequency reduced twice until generate and vibrate are stopped. In addition, pointing on axes causes generating low frequency tones. User also may get a feedback information about point: its coordinates and the information, if the graph contains the point.
Application is written in Java and it is available for mobile devices with Android operating system. The program has been tested by sighted and blind people. A positive feedback was given by testers, so application was done well.

More info:

I. Borowiecka, P. Skulimowski, M. Bujacz, A. Radecki, P. Strumiłło; “Interaktywna sonifikacja obrazów dla niewidomych – badania pilotażowe“, Przegląd Elektrotechniczny, ISSN 0033-2097, e-ISSN 2449-9544, nr 9, s. 98-101, 2016.

An optical data transmission system for the blind

LED DataMatrix anim code display
LED DataMatrix anim code display

Two dimensional data matrix barcodes are presented on the LED matrix display. Dedicated application for the Android based smartphones process a sequence of the modified DataMatrix barcodes.

More info:

A. Łutkowski, P. Skulimowski, P. Poryzała “An optical data transmission system for blind persons by means of a mobile phone with the Android operating system”, International Journal of Adaptive Control and Signal Processing, Article first published online: 1 SEP 2014, DOI: 10.1002/acs.2514, Volume 28, Issue 12, pages 1504–1513, December 2014