Steering a model by hand: gamepads, joysticks, MIDI controllers and keys
A live parameter can be moved while a run goes. Most people start with its slider, but a slider moves one parameter at a time with a mouse. Steering by hand binds a gamepad or joystick, a MIDI controller or the keyboard to live parameters, so that your hands can move several at once while you watch the plots or the machine. Every move is recorded, so a run steered by hand can be repeated exactly.
This page covers how bindings are made and set up, what is recorded and how to repeat a run from its record. For a short demonstration with real hardware, see the video and its write-up. The screenshots here use the keyboard, because it is the one device every computer has, and the Excavator example, whose four commands (swing, boom, stick and bucket) are live parameters made for this.
What can be bound
Three kinds of device, all through the same Bind button:
- Gamepads and joysticks. A stick or trigger is spread across the parameter's live range. A button can hold one value while pressed.
- MIDI controllers. Knobs and faders work like sticks. Pads work like buttons.
- The keyboard. A key held for one value and let go for another, or a pair of keys that ramp a value up and down.
A parameter must be live to be steered. That is a choice made on the parameter, which gives it a range (the excavator's commands are live from -1 to 1) and a slider. See Driving a model live for that.
Where bindings are made: Settings, Controllers
Open Settings with the gear in the title bar and choose Controllers. The page lists the live parameters of the model you have open, each with a Bind button, and below them the controllers the computer can see.

If you have a gamepad or joystick, check here first that it is the one you think it is. Every controller Konjugate sees is drawn as what it is, and its sticks and buttons move on the drawing as you move them. With two identical controllers, pressing a button and seeing which drawing responds is the quickest way to tell them apart. If a controller is not listed, "Not seeing your controller?" on the same page gives the steps for pairing one and says to press any button on it while Konjugate is in front, because a controller is shown only once it has been used.
Binding a control
Press Bind beside a parameter, then move a control or press a key. That control now drives the parameter. Konjugate then listens for the next parameter in the list, so you can bind them in order without pressing Bind again, and Esc stops listening.

A bound parameter shows its control as a chip, with Rebind to choose another control, Options... to set how it maps and Remove. Add another puts a second control on the same parameter.
How a control maps
Options... opens the binding's settings. What they offer depends on the kind of control. For a key the choice is how it maps:

- One value while held. The key sets the live maximum while pressed and the live minimum when let go, or any two values you give it. That is right for a switch and wrong for the excavator, where letting go should stop the joint, not drive it backwards.
- A pair of keys. Add a key for down... makes the binding two keys, one up and one down, that ramp the value while held. By default the ramp covers the whole range in five seconds, and the rate can be set. Let go and the value stays where it is. This is what the excavator wants, and what the screenshots use.
A stick, knob or trigger has its own choices: inverted or not, linear or finer near the middle or near the bottom, and a centred stick's centre at the middle of the range or at the value the parameter had. A reading within 8% of a stick's centre counts as the centre, so a stick that rests slightly off does not move the model. A button can hold a value while pressed, toggle between two values or step by an amount.
The same panel has Change control... and a checkbox, Only in "excavator". Bindings are kept in the app's settings, not in the project, and name a parameter by its symbol, so one binding works in every model that has that parameter. Ticking the checkbox narrows a binding to this model by its name.
Here Swing is bound to D and A, and Boom to K and I:

Steering a run
Close Settings and start an online run: choose Run, turn on Online simulation, set the pacing to Real time so that a second of the model takes a second, and press Start. The live controls open beside the timeline, and the keys now work. Click the canvas first, so that no field has the keyboard. Keys steer only while the window is in front and no field is being typed in.

In this run, D was held for about three seconds, then K for four, then A for two and a half, then I for two and a half. The cab swung round and the boom raised, and both came back down when the opposite keys were held.
Every move is recorded
When the run ends, the Changes panel lists what was done by hand, with one plot for each parameter that was moved.

A change is recorded by its size, not at every step. A moving control is recorded once it has moved by 1% of the live range since the last value recorded, and once more where it comes to rest, after a quarter of a second without movement. A press of a button is always recorded. That is why a few seconds of holding a key is 45 changes and not hundreds of tiny ones. The first entries in the Swing plot read:
| Time | Set to |
|---|---|
| 4.47 s | 0.04488 |
| 4.59 s | 0.09128 |
| 4.73 s | 0.1491 |
| 4.86 s | 0.2 |
| 4.98 s | 0.2463 |
| 5.1 s | 0.2978 |
Each is a live value of the parameter, exactly as if the slider had set it. The run's record is therefore the record of what was done by hand, whichever device did it.
Repeat it exactly
Repeat from the record, at the foot of the Changes panel, appears for a finished run. It offers a new run with the record's changes in its description:

The repeat starts a new branch from the run's first checkpoint and sends every recorded change again, each held until the run reaches the time it was recorded at. No controller is needed. The repeat is a branch of the original run, and it appears in the timeline's branch switcher, so the two can be compared.

I checked this on the run above. The original and the repeat each recorded 87 changes, with the same times and values, and every state at 30 s was identical. The documentation says the repeat is exact to the last bit with no controller attached, and a test in the engine checks it for values, a ramp and a waveform. In my run it kept the previous run's real-time pacing, so the repeat took another thirty seconds.
A run that was continued after it stopped is not offered a repeat. Repeating works by branching from a checkpoint, which is a feature of the desktop app.
What to use it for
- Feeling a system. A controller's gain or a valve's opening is much easier to understand when your hands move it and the plots answer at once.
- Showing or comparing. A hand-steered run can be shown again exactly, or its record can be played into a changed model, which tells you what the same driving would have done there.
- Driving a digital twin's sandbox. The twin's control room can try something ahead of the readings, and it follows the controls too.
Try it in the app
Two tutorials do this step by step and check each step, and both start from an example:
- Steer a model by hand binds W and S to the Position Controlled Joint's target angle and steers it with the keys.
- Drive with two joysticks starts from the Excavator, binds its four commands to the sticks of two joysticks or one gamepad (or to keys if you have none) and digs in real time.
Find them with the book icon on the left edge, or read more in the Tutorials post.
Things to know
- Bindings are kept in the app, not in the project. A project you send to a colleague carries no assumption about their hardware, and your bindings stay with you from one project to the next.
- A window reads its controls only while it is in front, except that the project window keeps reading them while an add-on's 3D window, such as the Pose Visualizer, has focus.
- The slider and a control share a parameter. The last to move wins, until the control moves again, and the chip says when the slider has taken over.
- MIDI. The Controllers page says whether MIDI controllers are available on the computer. On the one used for these screenshots it said they were not, so MIDI is described here from the documentation and was not tried.
Follow Konjugate on LinkedIn for updates, or subscribe via RSS.
Comments
Post a Comment