Alfimcev

Intelligent multimodal interface is user interface that can provide users with the opportunity to interact adequately to their modalities, recognize different situations to achieve users’ goals, create a model of user’s behavior and the environment in which it operates, modify behavior in accordance with the current state of the user and the environment. Design of intelligent multimodal interfaces is associated with overcoming a number of challenges and uncertainties by the developer. Therefore, they actively develop and study formal design methodology of intelligent multimodal interfaces which include a hierarchical set of interrelated formal models and operations on them, which can solve all tasks and related design automation. In this paper, the process-oriented approach to the formal non-deterministic design of intelligent multimodal interfaces using the concept of fuzzy running processes is presented. The method of selecting version of the draft of interface is considered. The language for visual representation of processes for the development of intelligent multimodal interfaces is described.

Research and development of human-machine interfaces, based on a recognition and visualization of multimedia information, is the forward edge in elaboration of modern mathematical methods and software. Developers of such interfaces try to use the natural human ways to communicate with computers through different modalities. Hand gestures of the user are very promising for building such interfaces. The soft-hardware system, that uses a video camera and a pico projector to create an interactive touch pad on a horizontal surface, is presented in the paper. The system creates the simple human-machine dialogue by organizing an additional way to communicate through the projection of graphical user interface. An original computationally efficient algorithms of object recognition and tracking, that provide the interactivity, are presented. The results of experiments and the model of the system are described.