Motion simulation

Motion simulation research can lead to all manner of projects – from live performance to measuring brain activity. At IISRI, we apply our research to a range of industry needs. We can create programs that assess ergonomics on assembly lines and design programs that identify people by their heartbeat. Motion simulation is a vast arena of innovation that can be adopted to make life-altering changes across the world.

The motion notion

Our Motion Simulation Lab is home to a range of mobile and industrial robots. We also developed the haptically enabled Universal Motion Simulator (UMS) – a flexible, high-fidelity motion system for dynamic, immersive training. 

The UMS represents the next generation in vehicle simulation, featuring a far greater range of motion, greater flexibility and more realism.

Dr Kyle Nelson – virtual motion master

IISRI's world-leading Universal Motion Simulator (UMS) is next-generation technology and Dr Kyle Nelson is Deakin's lead UMS engineer.

The gigantic robotic arm of the UMS delivers realistic accelerations and manoeuvres at high speeds in any direction, taking users on the ultimate, self-directed simulated journey.

This high-tech simulation is saving the aircraft, defence and automotive industries serious capital. The UMS is being increasingly used by designers and engineers to test new vehicle designs long before the innovations ever hit the production line.

'Once we load in the specifications of a particular vehicle type, the UMS can create the sensation and types of motion that drivers would experience if they were in the actual vehicle,' Dr Nelson explained.

Read more about the UMS

Visit Dr Nelson's profile

Research topics

Measuring depth accuracy in RGBD cameras

RGBD sensors project an infrared pattern and calculate depth from the reflected light using an infrared sensitive camera. In this research, the depth-sensing capabilities of two sensors are compared under various conditions.

The depth resolution decreases when the distance between the sensor and a planar surface increases. The Xtion depth resolution is better than the Kinect. The depth accuracy was also evaluated. The normalised histogram of the pixel bit values was calculated. It appeared that roughly 18% of the pixels maintain a single value, around 46% of the pixels alternate between two different levels, about 23% of the pixels have three different levels and the rest of the pixels have four or more levels.  This implies that the values of around half of the pixels fluctuate between two values. The entropy decreases when increasing the distance between the sensor and a planar surface, indicating lower depth accuracy.

Real-time ergonomic assessment for assembly operations using RGBD cameras

In many assembly operations, there are repetitive motions, uncomfortable postures and other ergonomic hazards. Ergonomic assessments increase productivity and performance by helping to prevent and reduce workplace injuries. 

The aim of this research is to use RGBD cameras for real-time ergonomic assessment in assembly operations. 

The accuracy and high sampling rates of RGBD sensors create an opportunity to monitor and alert operators if their current posture is risky. RGBD cameras are suitable for shop floor environments due to their portability, relatively low cost and rapid automatic calibration.

Methods used for RULA (Rapid Upper Limb Assessment) scoring: 

  • joint angles displayed on 2D tracked skeleton 
  • joint angles displayed on a virtual mannequin 
  • voxels displayed on a virtual mannequin. 

Our system provides visual feedback where the user's limbs are highlighted with different colours to indicate the RULA score for each limb.

Event-related potential analysis to identify the functional differences of the brain

The event-related potential (ERP) technique is a derivative of Electroencephalography (EEG), which measures brain activity during the cognitive processing of a sensory, motor or cognitive task. ERPs are obtained by time-locked averaging of EEG recordings for a specific sensory, motor or cognitive task.

The information flow or dynamic effective connectivity analysis applied to ERP data is a vital technique to understand higher cognitive processing and the functional differences under different events. Among other tools used in effective connectivity analysis, Granger Causality (GC) has found a prominent place. The GC analysis, based on strictly causal multivariate autoregressive (MVAR) models, does not account for the instantaneous interactions among the sources. If instantaneous interactions are present, GC-based on strictly causal MVAR will lead to erroneous conclusions on the underlying information flow.

Current research focuses on using extended MVAR (eMVAR) models for GC-based information flow analysis of ERP data. The eMAR model accounts for instantaneous interactions by adding the zero-lag component to the conventional MVAR model. 

The development of different adaptive estimation techniques is currently under investigation, with the goal of improved extraction of the underlying information flow using the ERP data. While improving the theoretical algorithms used in ERP based information flow analysis, these techniques are also used in a variety of applied research, including cognitive load assessment, development of brain-machine interfaces and rehabilitation-related studies.

Human identification from ECG signals

Biomedical identification and authentication using physiological modalities such as a person's face, fingerprints or gait have been widely investigated because of their significance in security applications. 

Recent research reveals that individuals can be identified from Electrocardiogram (ECG) signals as different individuals have different physiological and geometrical hearts, resulting in unique ECG signals.

Most previous methods extract feature representations based on individual heartbeat waveforms or fiducial points. However, accurately segmenting individual heartbeats or detecting fiducial points is an arduous procedure, especially for those ECG signals that contain noise. 

In this research, we propose to extract compact and discriminative features from ECG signals based on sparse representation of local segments. The proposed method is able to capture both local and global structural information and does not need to segment individual heartbeats or detect fiducial points.

Human motion analysis from video sequences

With advances in modern imaging technology and the reduction in the cost of electronic hardware, surveillance systems are now commonly installed in public places. Automatic analysis of human motion across a large number of video sequences is a challenging task. Human motion analysis algorithms that can model complex scenarios, and are simultaneously robust to viewpoint variation, noise and occlusion are highly desirable. 

This research proposed a general framework for the analysis of human motion in videos, based on the bag-of-words representation and the probabilistic Latent Semantic Analysis (pLSA) model. This consists of detecting human subjects in videos, extracting pyramid Histogram of oriented Gradient descriptors, constructing a visual codebook by k-means clustering, and supervised learning of the pLSA model for recognition.

Fault-tolerant force in human and robot cooperation

This work has investigated different strategies for the application of optimal fault-tolerant force within human-robot cooperation for the slow pushing or lifting of an object. Six different strategies were presented to optimally maintain a cooperative force, despite manipulator failure through a locked joint event. These strategies determined the post- failure cooperation of the faulty manipulator and the human.

Reliability maps for probabilistic guarantees of task motion for robotic manipulators

The aim of this research is to extend the concept of a fault-tolerant workspace to address the reliability of different joints. Such an extension can offer a wider fault-tolerant workspace while maintaining the required level of reliability. This is achieved by associating a probability with every part of the workspace.

Human passive motions and a user-friendly energy harvesting system

This research analysed existing methods of harvesting energy from human body motion and introduced a novel system for harvesting energy from abdominal motion. This method allows the harvesting to occur during completely passive motion, with a special focus on the user-friendliness of the harvester.

Pipe and well inspection systems and cable reels

The pipe inspection system is an efficient camera tractor for the inspection of pipes 200mm in diameter or larger. Its powerful drive enables proficient pipeline inspection over long distances and the electrically adjustable lift device enables optimal positioning of the camera in the pipe. 

In combination with a lowering device, the horizontally and vertically displaceable folding plug for the camera cable and the camera receptacle ensure a simple, safe introduction of the camera tractor to the sewer line. After inspection, the fast reverse speed makes it possible to quickly complete the procedure.

 The cable reel is a synchronised, fully automatic, motor-driven cable winch that holds up to 500m of cable. The traction regulating device coordinates the camera tractor and the cable winch and is developed for optimal operating conditions during sewer inspections.

Intuitive interfacing with crowd simulation packages

We propose a framework to use human gesture as an input to trigger events within a DI-Guy simulation scenario in real time. This could greatly help users to control events and avatar reactions in the scenario. 

We use the Microsoft Kinect device for motion capture and write our own plug-ins for gesture recognition and associate gestures with various commands in scene. 

The framework is a distributed system in which different modules are communicating and synchronising through data streams. This provides a scalable, loosely-coupled and highly cohesive modular framework where any component can be altered or modified without redesigning the whole system. 

The proposed framework could provide fast, affordable, reliable environments for real-time interactive crowd simulations. It supports direct user gesture input and allows for direct and collateral avatar reactions based on artificial intelligence. The gesture libraries provide most commonly used gesture recognitions and it could be easily expanded with more requirements.

Fusion capacity

The vast majority of image fusion cases take place by having high frequencies of information in a source image at spatial coordinates where the other image holds low frequencies. While this is quite common in fusing infrared and thermal images with visual images (at night), expanding image fusion to accept multi-source, multi-modal images raises concerns of saturating the resulting image and highlights the need to minimise the overlapping of high frequencies that causes fusion artefacts.

Natural scene imaging statistics suggest that finding a uniform image is close to impossible. Therefore, the fusion capacity of an image can be approximated by measuring the distance between the normalised histogram of the examined image and the uniform distribution using mutual information.

3D hydrodynamic analysis of a biomimetic robot fish

In nature, a fish adjusts itself to the hydrodynamic environment and becomes a perfect swimming expert. With the development of propulsive theories and robotic technologies, research on a biomimetic robot fish with high velocity, high efficiency and high manoeuvrability has been popular. It can also contribute to the development of more efficient propellers for ships or underwater vehicles.

At IISRI, we have designed a new free-swimming biomimetic robot fish with a biomimetic tail to simulate the carangiform tail, a barycenter-adjustor fordescending/ascending motions and multiple sensors. The robot fish can communicate with the outside by an information relay system on water. We also proposed a 3D computational fluid dynamic simulation of the biomimetic robot fish by Fluent.

User-defined function is used to define the movement of the robot fish and a dynamic mesh is used to mimic the fish swimming in water. The hydrodynamic analysis of the robot fish helps us get comparative data about hydrodynamic properties and guides us to improve the design, remote control and flexibility of the underwater robot fish.

Improving the kinematic performance of the SCARA-Tau PKM

One well-acknowledged drawback of traditional parallel kinematic machines (PKMs) is that the ratio of accessible workspace to robot footprint is small for these structures. The SCARA-Tau structure is a parallel robot concept designed with the explicit goal of overcoming this limitation and developing a PKM with a workspace similar to that of a serial-type robot of the same size. 

IISRI research shows how a proposed variant of the SCARA-Tau PKM can improve the usability of this robot concept further by significantly reducing the dependence between tool platform position and orientation of the original concept. The inverse kinematics of the proposed variant has been derived and a comparison made between this structure and the original SCARA-Tau concept, both with respect to platform orientation changes and workspace.

Kinematic performance evaluation of a novel PKM for solar cell manufacturing

Parallel kinematic machines (PKMs) are receiving increasing attention, both in academia and industry. One rapidly growing use for PKMs is in the solar photovoltaic (PV) industry, where a large number of DELTA robots are used. There is a rising demand for fast robots with larger workspaces than is currently available today.

The SCARA-Tau robot is a novel PKM with a large workspace that would be highly suitable for applications in the PV industry. Present research efforts at IISRI are aimed at evaluating the feasibility of using this robot in solar cell manufacturing applications. Adapting the SCARA-Tau robot to solar cell manufacturing involves finding optimal structural parameters for this application – an optimal workspace should have a large volume below the lowest upper arm of the robot, while the isotropic properties and dexterity of the structure should be kept high.

Effect of embedded attributes of wavelets and multi-wavelets bases on stereo correspondence estimation

Stereo correspondence estimation in one of the most active research areas in the field of computer vision and a number of techniques has been proposed and developed. Among the techniques reported, multi-resolution, analysis-based stereo correspondence estimation has gained a lot of research focus in recent years. 

The most widely employed medium for multi-resolution analysis is wavelets and multi-wavelets bases, but relatively little work has been reported in this context. Our algorithm uses multi-resolution analysis enforced with wavelets/multi-wavelets transform modulus maxima to establish correspondences between the stereo pairs of images. 

A variety of wavelets and multi-wavelets bases, possessing distinct properties such as orthogonality, approximation order and shapes, are employed to analyse their effect on the performance of correspondence estimation. In addition, comparative performance analysis of the proposed algorithm, with eight existing famous algorithms, is also performed to provide an insight into the capabilities of the proposed algorithm, as well as the potential of wavelets and multi-wavelets theories in stereo vision.

Super-resolution of a 3D scene

In many imaging applications, such as medical imaging and surveillance operations, it's beneficial to extract key details from captured images. However, these images are often of low quality and make it impossible to extract any meaningful information. 

Super-resolution is a method of enhancing images so that features of interest can be extracted in fine detail while still using low-resolution imaging hardware. Multi-frame super-resolution fuses information from a series of low-resolution images to create an image, or series of images, with a higher spatial resolution. As well as increasing resolution, this concept can be used to extend the field of view, remove moving objects and correct degradations inherent in the imaging process. 

Current super-resolution algorithms make a range of assumptions about the low-resolution images to simplify the scenario. Super-resolution algorithms generally use a simple affine image transform, permit only minute displacement between images and assume all images belong to a planar scene. In removing these restrictions, it is possible to greatly increase the number of possible applications for this technology. Allowing significant displacement between images and using a projective, rather than affine transform, means super-resolution may be used to create a high-resolution view of a 3D scene.

Image fusion algorithm and metric duality index

An image fusion metric does not suit all fusion algorithms. Some algorithms have to be evaluated with specially designed metrics. Through counter examples, we identify the compatibility challenges facing image fusion metrics with respect to a certain fusion algorithm. 

The proposed duality index depends on performing fusion experiments where we know exactly how the result should be. Developing such an index is fairly simple although it has its solid mathematical background listed in abstract algebra literature. The duality index depends mainly on estimating if the fusion operator measures the actual transferred data from a source image with a non-informative, also called a zero, image. The image fusion algorithm/metric duality index is then defined as DI0,:⊕x algorithm→Rwhere ⊕ is the set of all fusion algorithms and the set of all fusion metrics.

Colour map and histogram fusion

While single-domain image fusion systems aim to add information from source images into one fused image, the main objective of multi-domain image fusion is to incorporate captured information from sensors observing a certain phenomena from different viewpoints. This allows the observer to understand the whole situation.

However, not all sensors are as accurate, low cost, portable and readily available. In fact, some of these sensors have limitations according to the nature of captured data, power consumption and cost of the device. Most of these specially designed sensors adopt a colour map that best demonstrates the captured information. This is because most non-visual sensors are too complex to be equipped with large memory modules, faster codecs or modules to support higher bandwidth.

Optimal multi-sensor data fusion

Multi-sensor data fusion refers to the process of combining data from different resources to improve the quality of the information and accuracy of measurements. It's in the heart of a widespread range of applications, including military, smart buildings and bridges, satellites and industry. 

The main challenge with multi-sensor data fusion is that it's highly dependent on the quality of measurements obtained from each sensor. Since these sensors operate in real environments, there's no guarantee that the outputs of the sensors are accurate.

The main goal of this research is to incorporate the possibility of missing measurements or packets in the data fusion process and obtain the optimal data fusion for systems suffering from this problem. The research finds the minimum error variance of the resulting system state estimation using a Kalman-like recursive filter.

Robust filtering for uncertain systems

Kalman filtering is a well-known technique for state estimation of real-time dynamic systems. It provides the minimum variance error of state estimation when the system incorporates noises from the environment and the sensors.  However, the Kalman filters assume perfect modelling of the system and complete knowledge about the system parameters. In real-world situations, these assumptions are not valid and the outputs of the Kalman filters will not be robust.

This research addresses the problem of system modelling when the system under consideration suffers from uncertainties. It also provides the design of a new filter that provides the optimal filtering for systems under uncertainties without prior knowledge about the uncertainty values.

The uncertainties addressed in this research are the uncertainties in the modelling parameters and the uncertainties in the observations. The first kind may take place due to inaccurate modelling of the systems, linearisation of nonlinear systems or model reduction. The second kind of uncertainty is due to poor communication channels, unhealthy sensors, high manoeuvring of tracked objects or high noise in the environment. The results of this research include finding an upper bound on the estimation error covariance and finding the optimal filter parameters that guarantee the estimation error covariance to fall below this upper bound.

Robust rapid MRI

Despite the advantages of MRI, there are problems related to the use of MRI machines. It's a time-consuming process where a single scan may take 30 minutes, creating patient backlogs, plus the MRI scanning process is sensitive to patients' movements. In some cases it's hard to control the movement of the patient (often the case with children). In this instance the patient is required to do the MRI scan again, which increases the backlog.

MRI machines are very expensive, so increasing the number of machines is not especially viable. The practical solution would be to increase the speed of the scanning process itself. However, the speed of MRI machines is physically limited and cannot be any faster as long as the same number of measurements is required. 

This research looks at the problem of increasing the speed of the MRI scanning process using a novel measurements sampling technique that guarantees the minimum number of acquired measurements without compromising the scan quality. 

The idea is to find the best set of measurements that represent the whole set of measurements before starting the scanning process. This will eliminate any chance of redundancies in the measurements. The new data acquisition technique reduces the time of the scan by one third without introducing any noise. Higher speeds can be achieved in the cost of introducing additive noise. 

The results of this research also include guaranteed robustness of the image reconstruction in the presence of patient movements. 

The figures below represent the result of image acquisition from 65% of the measurements and the result after robustly filtering the output image.

3D sparse-feature model using short-baseline stereo and multiple view registration

This research investigates a methodology for generating a distinctive object representation offline, using short-baseline stereo fundamentals to triangulate highly descriptive object features in multiple pairs of stereo images. 

Several sparse 2.5D perspective views are generated and then registered into a single coordinate space. Having some prior knowledge, such as the proposed sparse-feature model, is very useful when detecting an object and estimating its pose in real-time systems, such as augmented reality.

Intelligent 3D programmable surface

Creating a highly programmable surface operating at relatively high speed and in real time presents many challenges. Such a system has applications in the field of optical telescopes, product manufacturing, and 3D-screens and billboards for advertising and artwork. 

This research investigates various system designs, modularity, programmability and system control intelligence. 

A simulation environment was developed to streamline system reconfiguration to translate complex mathematical functions into 3D shapes virtually, before being displayed on the physical surface.

Human performance measures in interactive virtual training systems

World-leading operations are increasingly relying on modelling and simulation to develop more efficient systems and to produce higher quality products and services. Modelling and simulation allows scientists and engineers a better understanding of three-dimensional and time-dependent phenomena, as well as providing a platform for predicting future behaviour.

Virtual training (VT) systems utilising advanced virtual reality technology is of growing importance in the areas of aerospace, engineering, medicine and education.

The perceived capability and technological affordance for enhancing human abilities to learn abstract concepts and complex procedural tasks of VT systems has led to a wide adaptation for training and fast-paced technological advancements. Despite this, the ways in which to evaluate efficacy of such technology are unclear.

In addition, how to better design such technology to achieve effective learning outcomes remains a significant challenge. In particular, consider the complexity of individual differences on human performance, adaptation and acceptance of new software and hardware devices evoked by 3D VT systems.

It's imperative to obtain a solid understanding of important elements that contribute to effective learning via such applications. This problem is addressed by developing a new evaluation method that focuses on cognitive, affective and skill-based learning dimensions. Results highlight the contribution of the method in analysing user performance, and understanding of the individual differences in learning ability and user experience within the 3D VT.

Vox Lumen – motion simulation in dance

As part of Melbourne’s White Night 2015, Vox Lumen: People into Light transformed Federation Square into an interactive world. It involved stunning abstract digital projections, dancer-driven live motion capture, and interactive content that tracked the movement of crowds across Melbourne's biggest night of arts and culture. The event was a combination of live performance and audience interactivity.

The fusion of live performance and technology featured dancer/choreographer Steph Hutchison, who wore IISRI's high-tech Xsens motion capture suit. 

This marker-less, full-body suit allows researchers to go mobile, taking a technology that is usually limited to a commercial lab setting into the heart of the city. The inertial-based suit measures the movement of the dancer using an internal gyroscope, so that every movement directly affects the content on the screen.

Read more about Vox Lumen