INTERLACE.AI is a digital interface for drawing that both augments and de-augments the expressive powers of the artist. The novelty of the interface is derived from a simple implementation of a classic AI technique.

I wrote a score for a performance using interlace.ai. With several collaborators, we performed the score at the residency SCORES: Propositions for Notating Performance.

Here's a video of the performance. I strongly recommend watching the video first so you can draw your own opinions, then watching the video accompanied by the "liner notes" below.

Here are some of the questions and realities we explored in the performance, taken from the score I wrote:

• How do interfaces mediate the relationships between humans and computational agents? A proposition for designing intimate interfaces, or interlaces.
• What does it mean to unwittingly use a human as an interface?
• What does it mean to perform an interface?
• Give agency to the subject, the observed, the algorithm: complicating machine obedience with machine disobedience, betrayal, aid, surprise. What counts as an ""artificial"" ""intelligence""?
• Replace the relationship between one person’s eyes and hands with the relationship between another person’s eyes and hands.
• An instance of pervasive digital surveillance; induce the awareness of digital gaze and observation. No: not just an omnipresent gaze, but the gaze of a hybrid creature of loving grace.
• Constrained communication through strokes and layers of digital lag and human response time.
• A design for improvisational and collaborative drawing—drawing as visual conversation.
• An interface that refuses to be used alone, that does not degrade gracefully in solitude. Reach through the screen. A tandem bicycle for the mind.

Here are some of the intended effects of the performance, also taken from the score I wrote:

• The audience experiences a conventional drawing interface, subject, and process that become increasingly uncanny and unexpectedly alive, culminating in a reveal that complicates the power dynamic between observer and subject, user and interface.
• The performance should ideally feel both magical and like a magic trick, which are subtly different things.
• The performance should ideally give audience members that feeling of being spooked as if in a haunted house—when you discover a ghost in the machine has been watching you.
• The performance should ideally include notes of confusion, wonder, and surprise.
• Ideally, after the performance, audience members should experience a lingering feeling of unease whenever using a digital interface.

Performance commentary

Act I

00:39 This audience member was actually a plant. This drawing went smoothly because we had rehearsed with the interface, the still life, and the mediator ahead of time.

Act II

00:40 This audience member was not a plant, but a genuine volunteer. However, the mediator had seen the still life ahead of time.

1:19 We had to add more objects to the still life because people were drawing faster than we thought they would.

1:29 This moomin-looking character is just one line in the drawing, but it's one of my favorite parts of the performance. We had asked the mediator to demonstrate an interface with predictive and refinement capabilities, so, eerily, this moomin appeared to be drawn on the screen before the illustrator had finished the stroke.

Here are the mediator's instructions from the score:

About halfway through the drawing, the MEDIATOR starts an improvised glitching process (see below [in the score] for possible glitches). Glitches should still be as fast as possible and close to the illustrator’s cursor.

In general, each individual unexpected action the mediator takes should be something that an algorithm or filter in a drawing interface might plausibly do. But taken together, the sequence of actions should become increasingly implausible: the illustrator should begin to suspect that the behavior of the interface, taken as a whole, could not possibly have been programmed in.

In this performance, in addition to predicting lines, we also had asked the mediator to act like a Photoshop brush. Where the illustrator drew a straight line, the mediator might draw a dotted line or a curly line. Some of the strokes were mirrored, and others were drawn normally. Finally, we asked the mediator to push the audience's horizon for "suspension of disbelief" by starting to act increasingly humanlike, e.g. by writing the illustrator's name.

1:33 The audience member's name is Rachel.

1:51 As the drawing becomes increasingly chaotic and unpredictable, I ham it up by saying misleading things like "I didn't program that in!" (it's true, I didn't) and "It seems that the Rachel woke up the algorithm." At the end of the drawing, I ask the mediator to come out of a back room by saying, "Would the algorithm like to reveal themselves?"

Acts I & II: behind the scenes

2:02 In case it wasn't clear, a human—the mediator—had been doing the drawings all along. This is the simple idea behind interlace.ai: every line you draw is actually "implemented" by another human. This other human, the mediator, sees every line you draw and can choose to draw whatever they want on top. (They can faithfully carry out their orders. Or they can disobey. Or they can help.) You, as the illustrator, see only the lines the mediator makes for you—none of your own lines.

2:03 In Act I, the mediator is asked to follow the illustrator's line as precisely and quickly as possible. Now you see what that harmony looks like.

2:14 Safari on the iPad is occasionally unable to distinguish a draw action from a highlight action, so the illustrator's cursor (shown to the mediator) is sometimes highlighted.

2:53 The mediator instinctively performs some autocorrection here. The second audience member had drawn a stray line, which the mediator does not reproduce.

3:14 The mediator later told me that it was very hard to decide on improvisational drawing tactics while the illustrator was drawing this fast.

4:04 The artist may be present, but the algorithm is absent.

Act III

4:11 For the final act, an audience volunteer was asked to draw the mediator from life. I was interested in seeing how the usual observer/subject power dynamic was complicated. Indeed what happened in the next few minutes was more beautiful than I could have imagined.

4:24 The mediator didn't like the illustrator's first few attempts to draw her. Each of them got impatient and cleared the canvas once (I cut one from the video).

4:52 I love the line quality here. But whose line is it anyway?

5:10 The illustrator and the mediator drew in silence for the first few minutes. (I hadn't given them instructions to talk or not talk.) Then I asked them to describe what they were seeing out loud as they were drawing it, and to negotiate the drawing verbally with each other. Here's one exchange they had while drawing:

I: Now your bangs are kind of coming up and over your face. Oops, a little low. That looks more like it, yeah.
I: I'm doing the other side of your hair. Coming up, up. This is fun. Oops.
I: Now I want to try some of your eyes, like some of your features. Not totally satisfied with that angle.
M: [laughs]
I: Now I really wanna make sure we get your nose good. So, coming down, it's almost like touching your... like all the way out to here.
M: All the way to there?!
AUDIENCE: [laughs]
M: I'm gonna stop here!
I: No, no, no, keep going, keep going!
M: No! [laughs]
I: Yes, it goes—it goes all the way to your cheek!

Here's the result of their negotiated nose territory:

In the image above, notice how the mediator has corrected some of the mistakes the illustrator made, like forgetting to lift their pen. The tear line was also the mediator's addition.

5:32 The lag got worse here. (We were using a tethered wifi connection in the backwoods of upstate New York, but there were also even more intermediate layers—human, software, hardware—at which lag could be introduced.) So it became essential that the illustrator and the mediator communicate verbally.

6:42 We ended the performance with a group discussion. Several people said they thought the mediator had too much power, and wanted to redesign the interface to equalize the power dynamic. From my experience playtesting the role of mediator, I found that it was the route of least resistance to play the algorithm: I would become hypnotized and enter a flow state when tracing another person's drawing. I would want to reproduce the drawing as accurately as possible, and I didn't often think to exercise my own agency.

In human-computer interaction and human/"AI" interaction:

• LAUREN: a human smart home intelligence (Lauren McCarthy)
• Wizard of Oz interface prototyping in human-computer interaction
• "Soylent: a word processor with a crowd inside" (Michael Bernstein et al.)
• Mixed-initiative creative interfaces
• Scrying Pen (Andy Matuschak) trained on SketchRNN (David Ha)
• The historical Mechanical Turk and the current Mechanical Turk

• The Brickcil (Doug Engelbart)
• Emma Brown, the mediator in this performance, had written the score to a dance where one person draws on a sheet of paper that is moved (by a second person) under the pen
• The Etch-a-Sketch
• Drawing using camera lucidas
• The Exploratorium’s "perform a simple action while looking only in the mirror" exhibit

• Muscle Plotter (Pedro Lopes)
• Tegaki, oekaki, paintchat
• Tablets
• Mechanical plotters and human plotters
• Surgical robots

• "Before code, beyond speech" (Lucy Siyao Liu)
• Sol LeWitt’s drawing instructions

• Doing redlines
• Art school drawing exercises, e.g. with your non-dominant hand, or not being able to take pen off paper, or drawing on a mirror

Let me know if you're interested in doing a co-drawing together on interlace.ai, using the software, or seeing its code. If there's enough interest, I'm happy to open-source the code and the score.

On the technical side, the software behind interlace.ai could use many performance improvements, server features, and interface features.

On the speculative side, I'd like to explore a variety of performance propositions for the relationship between the illustrator and the mediator. For example:

The illustrator and the mediator proceed line-by-line. Each time the illustrator draws a line, the mediator draws a line inspired by it or improvised off it, but it cannot be the same line. Each time the mediator draws a line, the illustrator draws a line inspired by it or improvised off it, but it cannot be the same line.

Practice together until you achieve groupmind, where both illustrator and mediator are in control and can produce a drawing in perfect harmony.

Thanks to Georgia Wall, Sarah Pitan, and Emma Brown for being willing accomplices in the performance, and thanks to everyone at SCORES and the Rochester Folk Arts Guild for feedback and community. This piece and performance could not have happened without them.

Lucy Siyao Liu provided helpful feedback on an early draft of this writeup, and Lea Albaugh gave pointers to related HCI papers. My understanding of "intimate interfaces" was influenced by early discussions with Robert Ochshorn and Michael Nielsen.