Researchers have built an AI system called Log2Motion that takes the physical movements of smartphone interaction — taps, swipes, scrolls — and thoroughly analyzes every minute of muscle stretching and twitching that makes those actions happen. They do this to better understand how our muscles interact with our phones and how much physical effort it actually takes to use them.
In all my time doing this, I don’t think I’ve ever come across a longer scientific journal name than Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. Buried inside this part of a title is a new way to measure the physical effects our smartphones have on our bodies.
For all the data tech companies are taking away from us with these things, they don’t seem particularly interested in data about the physicality of smartphone use. All they know is that you pressed a button. They often don’t know or care if it took an odd stretch of your thumb to reach the button that slowly develops a muscle tension.
Scientists say Doomscrolling messes with your body in ways you probably didn’t expect
Log2Motion simulates a digital human body using a detailed AI arm model with 63 muscle-tendon units. The system takes a series of educated guesses about how a real person would have moved to produce the recorded inputs. The AI even accounts for small human inconsistencies, as human movements are not naturally perfect. The researchers call this “engine noise”.
The real breakthrough here is something called “screen mirroring,” which lets this simulated AI body interact with real Android apps in real time. This interface between a physics-based body simulation and actual software did not exist until now. Because of this, a virtual finger tap or a virtual swirl of a polka ball in Pokémon Go can generate meaningful physical data for researchers to study.
To make sure they hadn’t created a good illusion of human movements, the researchers tested the system against real human movements, and the results matched well. The simulated movements fell within the natural variation you would expect if you had a number of people performing the same physical task.
All of this is an attempt to help app developers better understand how real, flesh-and-blood people physically interact with software. Instead of waiting for a flood of complaints to come in, app developers might one day be able to identify, for example, when a certain button is too far out of reach. Or when certain gestures require unnecessary effort before the app is ever officially released or even physically tested.
We spent so much time studying the effect of apps and smartphones on our brains, yet the physical actions required to make all that happen have been ignored for years… until now.
