I was delighted to return to Strange Loop this year with a talk about knitting machines and how they are programmed.
One of the nice things about Strange Loop (in addition to the gorgeous venues and of course the amazing people, from the organizers to the speakers and attendees) is that they prioritize posting high-quality videos of all of the talks.
So here’s my talk! It’s forty minutes long and covers knitting, machines, and a handful of languages for talking about both of them. I’ve posted a full transcript below the cut.
Transcript
Hi everybody. Just to be clear, there was apparently a room change, so this is “Languages for 3D Industrial Knitting,” not some web frameworks or something — webs are an entirely different textiles process. So this is knitting. Welcome.
I’m Lea. I work for that mouse, at Disney Research in Pittsburgh, Pennsylvania, and you can follow either of us on twitter.
This is a talk about knitting! There’s a lot to cover; we’re going to cover these things in particular: what is knitting, how does a knitting machine do it, and most importantly, most fun I think, is how can we talk about knitting and how a machine can do it.
So first of all, “what is knitting,” it seems like a good place to start. How many of you have ever knit anything before? Oh wow, that’s awesome, that’s a great number of people. Okay, just a little bit of catch-up for those of you who haven’t. Knitting is fundamentally a bunch of loops holding onto each other. And I just want you to know, given the context, that I did have to cut a bunch of loop puns otherwise I was not going to get through this talk.
This is one loop. Kind of. It’s not really a loop since you could just pull on it and it would just turn into a piece of string. But if we give it some buddies up and above, they’ll hold it in place. So now you see the one in the middle, the orange one, can’t go anywhere until at least the one on the top goes first. We can extend that out in two dimensions by giving them some neighbors on the side and some children above. And yes I do mean children above: in knitting, we talk about time as flowing from bottom to top, because when you’re making knitting, everything you’ve already knit falls down, because of gravity. That’s unavoidable.
This kind of looks like a knot, but a topologist would tell you it’s not really one, right, you could just pull the thread and it would unravel as you walk away. [laughter] Thank you.
The top one is the vulnerable one. That’s the one that is currently in progress. Hold onto that idea. We also call these loops stitches, by the way, I’m going to use those totally interchangeably and I won’t even notice myself doing it.
Something that’s really cool about knitting is because the same thread looping back on itself is what forms both the horizontal and the vertical connections, you can distort that stitch grid, right — so, if you overlap two stitches onto each other in one row, and then pull just one child through them, you’ve decreased the number of stitches in the next row. Again, time flows upwards, so on the left we have a decrease and on the right we have an increase.
So that’s how you can locally distort the fabric horizontally. There’s also a way you can sort of locally distort the fabric vertically which is called “short rows.” It means you don’t go all the way to the end you turn around midway and then you maybe finish that out by turning around again. So you see there’s more stitches in the middle columns than there were in those side columns.
If you apply decrease shaping and short row shaping to a shape such as a tube — maybe one that’s closed on the end, and then you stuff it and put cute eyes on it, you’re going to get a cute little creature like that. So that’s sort of the foundation of how you might be able to do complicated shaping and knitting. By the way these little buddies are way better in quantity. I’ve got some of them up here; you can come play with them after the talk. Come talk to me.
Ok so that’s knitting, right, but how does the machine do it? Right, like that sounds like a really complicated structure to make and it is kind of complicated. But I do want to give a little bit of context. We have had knitting machines for a long time — not nearly as long as looms, which are for weaving, but we’ve had knitting machines since 1589 which is when William Lee invented the stocking frame, right at the beginning of the Industrial Revolution. And so these stocking frames would be found in like cottage industry workshops, so they wouldn’t be like a home so much as like a home industry, just like a loom or spinning wheel. By the end of the Industrial Revolution these knittings machines were seeing both industrial and home use. Circular sock machines (the thing on the left) were kind of like a sewing machine of that era, which is to say, very expensive but a major time saver for people who did have one and wanted to make their own things. Kind of like a 3D printer today, right, so not everybody is going to have one.
But moving along in time these flat knitting style machines which is really what this talk is mostly about, started being marketed as a homemaker tool from roughly the nineteen-forties onward. Consumer knitting machines reached a technological and sort of popularity peak in the eighties and nineties, culminating in computerized models that could store patterns. Previous ones used punch cards. These electronic ones are the ones that you’ve maybe seen recently on the Internet because they’ve been what the maker movement has been working with. Just some examples of that because I think it’s important for you all to know that these things are out there.
These are two software modification programs, projects for knitting machines: Becky Stern and Limor Fried on the left building on Steve Conklin’s disc emulator work that’s for the Brother KH 930e model; and the All Yarns Are Beautiful project, on the right, which is an Arduino override of some older brother models. There’s also some open source [hardware] knitting machines which is cool; you can printing your own machine. On the left, a circular knitting machine like the sock ones earlier called the Knitic project; and on the right, Kniterate, which is a flat-bed knitting machine, very cool .
Meanwhile, of course, industry had been perfecting the knitting machine as well. Much more reliable, faster, more stitches per inch, and most excitingly, the ability to move stitches around – which as you may recall, or you may be sort of thinking, before, those increases and decreases, you need to be able to remove stitches around. There’s a picture of our knitting machine in the lab in Pittsburgh. It happens to be made by the Shima Seiki company. They call this kind of machine a “wholegarment” machine. There’s a handful of other competing companies, notably Stoll which calls their equivalent “knit & wear” — emphasis on “you can make a thing and it’s like done when it comes off the machine.” Pretty cool. So want I want you take away from that little interlude is that knitting machines exist in the world. This is not the future; this is the exciting present.
They’re very fast, they’re very flexible, they can work with a huge variety of yarn — so, yes, conductive yarns; also monofilament, thermoplastic yarns, elastic yarn, etc etc. Textile stuff is all around us — clothes, of course, you probably spent a lot of your time clothes and around furniture, that’s pretty typical. There’s also automotive and aerospace upholstery, medical and assistive technology, soft robots, toys. (My company cares about toys and things like that.) Right, and knitting machines in particular, as opposed to cut and sew and weaving processes, can make things that are one-off unique objects without necessarily having to set up an entire production overhead.
So how does that work? Right, ok, so this is our machine. In the center there, just past the safety cover — because it is an industrial piece of equipment — there’s a shiny wide row of needles. If you’ve knit by hand you’re used to needles as being essentially a straight stick. On a knitting machine, they’re hook-shaped and each one holds onto some of those loops at the top — the vulnerable ones, the ones that could be unraveled easiest — and each needle also has a slider which can slide forward to close off the hook. And the machine also has some yarn carriers which are abstracted here as sort of triangles but they’re basically a way of holding the next bit of yarn where we want it. You can think of it as kind of like a yarn extruding device in terms of what is getting us.
A full machine is going to have hundreds of these needles next to each other in a long row and we call that a “bed.” Our machine is a “v-bed” knitting machine which means it has two beds, and instead of tilting forward like this, two of them are tipped upwards and they meet kind of like this: almost meeting in the middle in an inverted V, which is why we call it a v-bed machine. So you see it from the side on the left and then you you see it from the top on the right, which is a notation I’m going to use later in the talk.
Ok one more thing: the needles obviously do not just passively hold loops, because that would be really boring knitting machine. They also all move up and down to form the new stitches and since there’s so many of them — on ours it’s like 600 on bed — it would be impractical to power them all independently. So there’s what we call a “carriage” which is essentially a moving cam path that slides by, grabs them, pulls them up, and pulls them back down again. So that’s what you’re seeing moving back and forth in the video.
Ok, so what are the needle doing? Right, each needle is capable of performing four basic operations. First one sounds kind of obvious: it’s the knit. So if a needle is holding onto a loop — a stitch — it can push forward, leaving the slider behind, and the existing loop is going to push back onto the slider. While it’s forward, the yarn carrier is going to drop a little bit of yarn in that hook, then the needle is going to pull backwards, pulling that yarn into a new loop and simultaneously dropping the old loop off of the nose of that closed hook. So here’s an animation of that happening to a bunch of needles in a row. And you can kind of see the cam path, the implicit cam path in how the needles are moving next to each other.It’s very pretty and this is what I look at all day long. [Audience member: “Don’t stop it!”] You can watch our videos on the internet.
Alright so we have a tuck operation. It’s similar to knit in that it puts some new yarn into the hook; unlike the knit, it does not drop the old one. So this is a way you could end up with more than one loop on a needle — it’s one of several ways you can have more than one loop on a needle.
Okay we also have an operation called the transfer. This one’s really exciting; this one moves a loop from one needle to its cousin right across on the other bed — because there’s two beds so every needle’s got one just across the bed gap. So it kind of slides up, slides the loop onto the slider, hands it over and the other hook grabs it. Pretty cool, but I just said it only transferred to the one right across, right, because it’s sort of this dumb movement, they just slide forward and backwards, which would make this not a very useful operation if we did not also have another operation called “racking.” So this is rack in the sense of “rack and pinion” and it means the entire back bed can slide laterally. So you can think of sort of a default value we’re like needle f[ront] 0 is lined up with needle back 0, and you could slide all the back ones leftward so that needle front zero is aligned with needle [back] 1. Ok so if you combine the transfer and the rack operations, that’s how you’re gonna move stuff around.
Here’s a little animation where you see two stitches getting knit and then one of the loops gets handed off to the other bed, the other bed racks over, and hands that loop back again. That’s cool because now we’ve got two loops stacked on top of each other, which is the foundation of how we do that decrease that we saw earlier. Similarly if we’d racked the other way we could have dropped the stitch the other direction and then we would have increased. Pretty cool.
We have a 4th operation called the “split” which is kind of like you’re simultaneously knitting and transferring at the same time. So that uses the other bed because you’re transferring the original loop over to the second bed and at the same time the yarn carrier’s laying in a bit of yarn and pulling that into a loop. This is helpful because otherwise you don’t have any way of essentially retaining a pointer to that first loop, because otherwise it would just drop off and you never be able to get back to it again.
Okay, four atomic operations. Sounds really simple, and you should keep that simplicity in mind because this is what I’m going to start talking about complexity.
This talk is about languages for knitting, so I wanted to start off with talking about human languages for knitting, which is to say, how do human knitters talk about how knit objects are made. Probably the most common language for hand knitting is what a lot of people call Knitspeak, and it’s kind of a loose collection of commonly-used abbreviations and some light expectations about syntax. You can think about it as a lot like a cooking recipe, and just like a recipe, knitspeak focuses on operations, not necessarily on results.
There’s no universal standard list — lots of regional variations different emphases especially depending on what kind of thing you’re doing, so most patterns will often actually start with a little glossary like this. This is the list of standard abbreviations on knitty.com which is an online knitting magazine. (It is pretty cool.) This is kind of a representative chunk of it. At the top you see “k,” that just means “knit,” you’re gonna see that pretty often typically. And a couple down you see “k2tog,” meaning overlap two stitches together and it through both of them — so that’s that decrease, gonna keep seeing things like that. And the one on the end there is a kind of increase. There’s a couple of topologically distinct ways to increase stitches; they’re named by how they’re made, so in the cooking recipe that’s kind of like saying “steam” or “fry” as opposed to just like,“heat.” Another thing to notice is that knitspeak is kind of a hodgepodge; it mixes together stuff like how big your needle is supposed to be and how much yarn you’re going to use and entire subroutines like “make a bobble” and things that are variables like “main color” which could be whatever color is your favorite color today. Something that’s pretty universal in knitspeak is “rep from *,” in other words it’s a basic loop. And these can be “for” loops, so like sometimes it’s, you know, “repeat three times.” They’re much more often “while” loops, so it’s “repeat until the end of the row.” That’s can actually be really convenient as a checksum because they’re expected to repeat like an integral number of times with no more or fewer at the end so if you end up with that not being the case you probably dropped a stitch or something. Probably need to tear that row out, wah. Another thing to call out is this one “WS” which means that the next chunk of knitting are applying to a wrong side row. Ok what does that mean? The knitters in the audience know a lot of people — most people — knit only in one direction — most people specifically knit right to left. So you’ll knit a row over, turn your entire piece around, and then knit the same direction again. Which is all well and good, except it means that half of the time you’re looking at your piece from the back, and all of the operations you’re making are mirror image operations. Which makes sense when you’re making it. Its kind of opaque though when you’re reading it.
So a lot of hand knitting patterns come in the form of: charts. Which are often used in combination with knitspeak so you’ll have like a chart for just one part of the pattern. This a basic charted pattern for lace. You can see the repeat is the heavier vertical lines, so that is the part that you would repeat out if you wanted to make it wider. This is a lace pattern so it’s actually only showing us every other row and implicitly you would do a row of the same kind of stitch all the way across in between. But otherwise, even if you were seeing all the rows, they would be shown from the front. The symbols are intended to kind of look like what they mean so the the slash and the backslash mean decrease, like merge some things together, and the the sort of A with the leg on it is “decrease [three] things together” and the O is essentially “leave a little hole,” so it’s a very visual way of seeing what you’re gonna get. Now unfortunately it’s kinda limited by the fact that it’s a rectilinear grid. This is a knit swatch of that same pattern, the “long leaf lace” by Barbara Walker (she’s amazing) and you can see that the actual knit rows tilt up in the middle because of the decreases. I’m only mentioning this because I really like this variant notation that was written by JC Briar of stitch-maps.com. It actually warps the grid to show you these distortions and it parses these charts from knitspeak. So it’s an entire knitspeak parser that makes these diagrams and it even shows you the the course and wale directions if you want it to. As far as I can tell, this is a really recent work, so, knitters in the audience, if you didn’t know about this, this is really cool.
But so all of this chart stuff is really cool for talking about knits structure, sort of, operations really, but it can only go so far in helping us to talk about what a machine is going to do. Because machines can only do four things (sort of) and for another thing human leaders often knit on those long needles and their stitches are going to just slide back and forth and it’s really easy to put in an increase or decrease. Whereas on a knitting machine, you have to allocate everything its own needles. Harder. So how does this work? As I said, these exist in the world; people are programming them. It turns out that each manufacturer has its own control language, of course, with varying features. Ours was made by the Shima Seiki company so we use their language. It’s just the one that I’ve worked with. And it’s called Knit Paint and it looks like this.
It’s… as you can see, it’s a simple imperative language without conditional branching and with only rudimentary loops. So you may have noticed it’s not textual. This is not an esolang. Or, I mean, it is esoteric but it’s not invented language, like just for the purpose of being esoteric. This is a production language. So in the middle we’ve got a kind of needle by needle set of instructions, where the needle position is encoded by the X-position of the pixel and the construction time is encoded as Y.
Each line from bottom to top — again, bottom to top; time flows up — is a single pass of the carriage, either left to right or right to left. It’s a little weird because both of the beds are collapsed into one line, but the operations are distinguished by color. Of course, the front bed ones are a little bit pinker and the back bed ones are a little bit bluer and greener usually.
So what we’re seeing here is a basic tube. I happen to know that we’re knitting over to the right on the front bed which is the kind of salmony pink color and then we’re getting back again to the left on the back bed which is the mint green. So if you knit one way on the front and back again on the back, you’re knitting in a spiral, you’re making a tube. Off to the side are option lines, so those are setting values that apply to the entire carriage [pass], so the leftmost one which is kind of like a teal and white stripe is telling us which way the carriage is moving, except for the red ones, that means the carriages moving at all. It’s moving the yarn carrier. A few to the right of that you see one that sort of teal and army green; that’s telling us what yarn were using, so teal of course means six, so that means carrier six, and army green is 255 which means the “null” yarn, so no carrier there. The rack value is set by two of the lines on the other side, of course, two of them; they interact in a very bizarre way. Other stuff: like that kind of pale pink means there’s an extra little press down mechanism that’s going on; the green and kind of like dull green in there is how big the stitch is going to be. It’s all the stuff, you know totally obvious, very reasonable.
Something that’s interesting about this language is that it provides for macro expansion. [laughter] You’re ready, I can tell. So this is an example of a knit tube with a bind off. So in knitting, as I said that top row is vulnerable, right, that the last loop is always vulnerable until something else is holding it. So when you want to finish a piece of knitting often what you’ll do is knit a loop, overlap it onto its neighbor, knit one loop through that, overlap that onto the neighbor, and so on and so forth. And so you have only one vulnerable loop at the very end and then you can tuck the yarn tail through that and then finally, topologically, it is the knot that it has always dreamed of being.
So on a knitting machine, because every time we do a transfer at a different rack we have to actually break that into its own carriage pass line, that tends to look like this diagonal thing. So the first part of the diagonal (again, time goes up) is moving loops rightward and then we’ve gotten to that edge of the tube and we’re coming back on the back bed of the tube leftward again. That would be really tedious to write, so fortunately we can write a macro. This is what that would look like in KnitPaint. So at the top we have what’s called a base pattern and it kinda looks like the other one except that instead of having that whole complicated bindoff, it’s got just one row of beige at the top. And on the bottom, we define what beige means. And if you can see that it’s that same beige where the arrow is. Yeah it’s very easy to get the wrong beige by accident. And then there is an elaborate mark-up notation for how this is going to translate to something that is spread out across two dimensions. So at the top there, those green pixels mean that as we’re moving rightward over by one because one is green colored — or sorry, over by two because two is green colored — that means that we’re going upward by one and to the right which is red color on the other arrow. It.. I.. This is something I’m not very good at. But there are people who do this, like for living. And so then you tell KnitPaint that that’s your base pattern and that’s your package and it processes it together and it gives you that sweet bind off that I showed you before.
So that’s great, because now you can work at a slightly higher level. Actually, I’m joking a little bit, or like, I’m overplaying it, because really knit programmers will typically start with some software provided parametric templates like these. This is a template for making a glove; you’re like, oh I probably want five fingers, and like a thumb that’s set in, and my fingers are this long or whatever. And that stuff will then, by KnitPaint — this is sort of the Clippy of KnitPaint — and that stuff then gets turned into something that looks like this, which is just a really big version of what I was showing you earlier. So we can zoom in, and all that stuff along the left-hand side was the packages, so this is like a lot of packages involved, and then the right hand thing is the base pattern that is going to get processed with those packages. You can see that is still kind of fiddly looking. This is what it looks like once its expanded. I bring this up because you can see the rudimentary looping I mentioned earlier. Those would be very short stubby fingers of a glove, except where those arrows are pointing there’s a couple of colors in option line Right One which you can allocate to a certain number of repetitions at compile time. Another cute feature of the macro expansion thing is that the stuff along the left edge is what color those macro packages were in the original pattern so you can kind of see who to blame when things go wrong.
So this has the advantage of being an actual production language. That’s good. It can be compiled to real control over the machine, and it’s got a fairly vast library of convoluted templates and macros. But you’re probably not surprised to hear it’s really fiddly to work with. I have had problems — I actually spent an afternoon debugging a problem where I was using the wrong teal — I was using color six instead of color sixteen; it took me a really long time to figure it out. And while the KnitPaint environment does have some light compiler warning and debugging — like, linting tools really: it’ll warn you if it thinks you didn’t mean to drop a loop — it’s basically a paint program. So you have, like, a pixel pencil and, like, flood fill tools, and things like that. And the macro stuff tends to be really sensitive to parity, so if you were writing a macro for something that was an odd number of stitches and suddenly you wanted to use it on an even number of stitches, you probably need to rewrite the macro. So that’s not optimal.
But I said earlier that there really arguably only like four things that the needles can do. You know, these things, right. So that suggests that we could strip our concept of the knitting machine’s operation down to something more like an assembly language. We could embed that in the language of our choice — turns out to be javascript; don’t argue with me about this [laughter] — and once we’ve done that, we can have a foundation to build more attractions on top of. Okay great, that sounds good; let’s do it.
Cool, so this is kind of what our assembly language looks like right now. So knit, for example: remember I said it it happens at a particular needle, so we’re probably going to need to include in the specification which needle we’re talking about, and we could encode that as the bed and the number, so it’s like “front one” or whatever. We need to know what yarn we’re talking about — most knitting machines have more than one, and also there’s like the null yarn — and we also do need to know which direction the stitch was formed in, because that applies with the carriage and which way the yarn is moving across the needle. Tuck: very similar.
Transfer: that’s an operation between two needles, so we need to specify both needles. We kind of don’t really need to specify both needles, right, because we could either have left the second needle or the rack amount implicit. We didn’t. We both explicitly specify the second needle and we specify a rack, which does mean that sometimes you can have invalid transfer operations. So if you said we’re at rack zero and I want to transfer from “back 1” to “front 0” that cannot happen; that’s not physically possible. If you rack to -1, then you can transfer from “back 1” to “front 0.” So it’s a little over specified, and we could have left rack value implicit, or we could have left the second needle implicit. One reason we do include the rack as an explicit operation is that changing the rack of the machine is a constant time operation, so it helps us think about that, and improve our knit-time efficiency a bit. Another reason is that the rack assignment doesn’t only matter for transfers; it also matters when you’re making new stitches, because any time you cross the bed gap, you might leave a longer tail of yarn if they’re not aligned, for example. And of course errors thrown this way are typically indicating there’s some other fault in the code somewhere. Okay, so knit, tuck, transfer, and then split is, as I said, it’s basically a knit at the same time as a transfer, so it needs all the same arguments as knit and transfer.
Ok so now if we wanted to do that thing like I showed you earlier, like we’re going to knit some things, transfer them back, rack over, transfer them forward; that’s kind of what it might look like in assembly language, right? Knit some things, transfer them, etc, etc. And you are all programmers, you’re like “oh that’s awful, that’s tedious, let’s please at least write a loop.” Okay, good, loops; we like loops! (Some loop puns remained.) So this is the kind of thing you’re gonna do right like starting at the first one, run through the last one, knit some things at that needle. Obvious. I do this a lot at my job, right; I write a lot of knit jobs. So of course I started writing the same functions over and over again, which of course means that I started to distill them into my own sort of personal library. Also when I say my own personal library, I was told that this is where I’m supposed to say “embedded domain-specific language.” So pretend I said that instead; it’s cool or something; I don’t know.
So my library is called Nancy. It’s called Nancy because a knitting Nancy is another name for a knitting frame and this is the framework for knitting, and I’m told that pun-based development is as cool as embedded DSLs. So here’s a simplified taste of what Nancy looks like. So this is a function that knits some rows; we obviously spend a lot of time doing that. It takes in a description of a fabric, which in the Nancy library could have a whole lot of parameters to it; it’s at least going to have the left edge and the right edge of that fabric. It can also store things like what kind of knit texture; what kind of rib or lace texture is going on; whether it’s flat fabric, which is to say, on just one bed, or a tube fabric, on both beds; whatever. In this case we’re only using the left and right edge. It’s a simplified version so I’m assuming this is a flat fabric. So, for however many rows you want to do, knit from the left to the right and then knit back from the right to the left.
I first started working on this library when I was making these little modular stuffed creatures and I designed it with kind of an Exquisite Corpse approach. So if you’ve played the dadaist game Exquisite Corpse you know you kind of like draw some lines on some paper and it doesn’t really matter what the lines do between these connecting lines; as long as everything ends up back at the connecting lines, you’re golden. So that’s how these little guys work: you know, there’s like some leg modules, and there’s some like torso modules, and there’s some heads, and all of them return this fabric type which includes things like left edge and right edge and whether we’re a tube or whatever. These guys in particular they’re all the same width at the joins and they’re all tubes at the joins. So that’s pretty cool, I mean it’s like, at least that gives me, you know, a way to not stress about those connections. So knitRows doesn’t change what’s happening between the connections but some other functions might, right; so like, moveStitches, it does what you think it does: it takes some stitches, it transfers them over and back again, and so it actually means something then, that I’m returning where the new edges are. So if you wanted to do that thing earlier with Nancy in particular, we could do it like this: we would define a little fabric object, we would knit it, and then we would maybe define the part of the fabric that we want to move over, and we could tell that to move, and then we would like square our books with making the old fabric’s edge be the new fabric’s edge. It’s not perfect but it’s a lot better than writing like “knit one, transfer one over there, and back again, I guess.” So that’s Nancy. I do use it a lot, like all the time at work, but there’s a major limitation here so I was saying, transferring stitches. Transferring stitches is one of our biggest problems. In Nancy I “solve” this problem by only doing a couple of very specific kind of transfer. All Nancy fabrics are symmetric on the front and back so if you’re doing a tube the left edge of the back face is the same as the left edge of the front face. That means if we’re decreasing the front face, we’re inherently also decreasing the back face. Aesthetically, this is the kind of thing that hand knitters also do, so it kind of looks right, but that’s a bit of a cheat answer because practically, it’s also just really hard to not do that. And to give you a taste of why it’s hard, I want to tell you about some of the constraints involved in transferring.
So, as I said, the machine can transfer a stitch across to the other hook, right, but when we transfer a stitch, we transfer everything on that one hook. So if we wanted to move stitch A on to stitch B, if we put it on stitch C’s needle, now stitch C is coming wherever stitch A’s going, right, we’re going to end up having both of those on top of stitch B, which may or may not have been what we intended to do. Fortunately we actually also have one more transfer option which is that we can transfer the stitch to the slider instead. So the slider can actually move forward past the hook and it can grab some stitches there. So that’s cool, we get a little bit of a reprieve. Unfortunately — so that would look like this, so this the slider has moved forward and we transferred to there so there’s like, one more bucket to put stitches in, but that’s still only two buckets and if we put more than once in the slider those two are again inseparable. Also there’s some quirky mechanical rules, like we can’t knit any new stitches while anything’s on the slider; stuff like that. But it does give us a bit more flexibility; that’s good. Another thing we can’t do, or more accurately, probably don’t want to do is rack the bed really far. If you have some stitches on the front bed and you have some stitches on the back bed that are connected to them there’s inherently a yarn connection between them, it’s a physical piece of yarn, if you pull it really far, it will break — it’s easy, I do it all the time. Ideally you wouldn’t do that. In fact, ideally you would only pull those loops as far as like, the final position it’s going to be in, or where they currently are, whichever is the bigger of the two. So another thing: it’s physically impossible, you cannot pass a stitch through another stitch like that. But you can tangle them up like that. You probably don’t want to. This state is also really prone to yarn breakage. This maybe seems easy to think about when it’s only six stitches in a nice little ring, but it gets out of hand really quickly. It would be nice to automate this. It turns out that’s hard. Maybe NP hard? It’s hard to tell.
So I’m going to talk a little bit about another project, which is a compiler for 3d machine knitting. So this is my team’s SIGGRAPH 2016 technical paper; that’s a my boss, Jim McCann, myself, Vidya, April, Wojciech, Jen, and Jessica Hodgins. You can read our paper online: disneyresearch.com. And the deal with this paper is we present a constrained set of primitives — so it’s just tubes and sheets — and they have some parameters in both their final form and the production instructions. This, by the way, is the first language I’ve talked about in this talk that is not like a cooking recipe: it’s not imperative, it’s declarative.
To be clear about the relationship between the projects I’m talking about, both Nancy and the compiler project are built on top of the knitting assembly protocol but they are otherwise separate projects. So working on Nancy did help me think about what I would want in the compiler but they solve the problem in very different ways. Also in the interest of full disclosure, we do actually still compile our knit assembly to KnitPaint.
So knit objects in this system are represented as a series of tubes or sheets; they’ve got heights, diameters, different diameters, bendiness. You may recognize the different diameters and bendiness as increase/decreases and short row shaping. Cool. They can also be attached to each other along their top and bottom edges and along the front or back beds. So gluing is actually — it’s hard to not do. If you leave some stitches on a certain needle and then you knit back on that needle later, inherently those things will be attached to each other. So that does rely, however, on you putting them on the same needles. So the compiler therefore kind of uses a notation similar to KnitPaint, which is that we think of a two-dimensional space with needles being one dimension and time as the other. So then you, the user, can schedule these fabrics in this space by moving them around. If you’ve got two fabrics on top of each other that won’t knit because you’re saying, “use these same needles for two things at the same time.” Can’t be done. Fortunately we also have these scheduling parameters that don’t actually change what the knit shape looks like but they do mean “as your knitting this sort of maybe move it over on the bed,” for example, so that you can ensure that some things connect where they might not otherwise have.
So here’s an example of a fingerless glove made in this system. There is a textual version and there is sort of like a graphical interface version; these things are equivalent; they’re both input to the system, not output. And again we’re seeing this in construction time, so on the graphical version you see that there’s like gray areas — those are not knitting, those are just where there’s stuff on the needles that is waiting to be picked up again. So we’re knitting all the finger tubes across and since the same yarn, we’re like actually breaking it and then moving over and making a new one, and then picking them all up, adding a thumb, picking that all up, and then decreasing inward to the wrist. Ok, here’s what that looks like. You can actually maybe see the decreases, I dunno, maybe the front row can see the decreases. But the compiler put those in automatically.
The compiler, as I said, allocates fabric to specific needles at specific points in time, which means it needs to be able to transfer things. So, you know, transfers are hard. My ad hoc method — my artisanal method — doesn’t really handle all the possible things you might want to do, and the compiler, of course, does not either, but it does offer a complete (proof is in the paper) [if] inefficient solution to the transfer planning problem. Remember the in the compiler all the fabrics are either tubes or sheets, which means that if we look a slice of it on the bed, it’s always going to be a cycle, like the thing on the left; it’s never going to be a thing like the right, which is like a figure eight. So because of this, we know that the only cross-bed connections on the left and right edges and we also know that the only yarnwise connections are to the neighboring stitches along the cycle. That’s good because that allows us to do our little algorithm. So let’s say we have the thing on the left, it’s a six stitches in a ring, and we want to end up with a thing on the right. I’m going to start going through this kind of quickly because I’m starting to run out of time — so, real quick, what we’re going to do is we are going to pick a direction and collapse all of the other bed’s stitches onto that, so for example we are going to take that stitch, move it over there, take it’s buddies, move them over there; that was collapse. Now we’re going to do a stretch phase which can sort of add like global length changes to the whole cycle, so that was good because we needed to kind of stretch it a little bit; so those ones are going to go there, those ones are going to go there, and that’s the stretch phase complete. And then we’re going to do what’s called expand which of course expands it back out onto two beds. So this case is easy; we’re almost ready to go; everything just goes back. Great; we’re where we wanted to be! Fantastic! Again, this is a very simple case. This is accomplished in the compiler with a dynamic programming approach; it’s broken into those stages that I mentioned, so: collapse, shift, and expand. Each stage is greedily attempting to get stitches as close as possible to their final position. As in, the objective function is just a sum of distances to the final positions. The process is not guaranteed to be complete after one iteration of collapse/stretch/expand but it is guaranteed that it will eventually get there.
Here’s a payoff teddy bear shot. It’s Duffy Bear, and he’s wearing some clothes that I designed for him in the knit compiler system. There’s also this snake, who is kind of my favorite. You can see he’s basically just a long tube with aggressive short row shaping. And I actually wrote that as a script on top of the compiler, so it produces a series of short row specifications increasing in radius and rotating around the tube. Also, I did not make this gif; a stranger on the internet made it and it makes me really happy. And they titled it that! So that’s great.
Ok so unsurprisingly, we have barely touched on the breadth of things that knitting can do. First of all, one thing that’s pretty obvious is we could use better interfaces. The thing on the left does not look like the thing on the right, not really. So we could use better predictive rendering; that would be awesome. Another thing we did not talk about is efficiency: knit time efficiency. Knitting machines are really fast, especially if you’re used to 3d printers — so much better — but our code is not as efficient as it could be. Especially the compiler; as I said, it’s kind of wasteful with its transfer planning; it could be a lot more efficient. And another thing like all of you knitters in the audience — I saw there were a bunch of you — you probably noticed that I didn’t talk about anything other than plain stockinette fabric. Knitting machines can do a lot of other things. So there’s laces, there’s color work, there’s ribbing which is kind of just “you have to be careful which needles you’re using” and we didn’t talk about that at all. These things are kind of easy to talk about when you’re talking about rectangular swatches but how do these things map onto cool shapes like J-30. Like what would a lace J-30 look like? There’s aesthetics also to be considered and UI tools and things like that.
And that’s all I’ve got, so: I’m Lea, I work for Disney Research in Pittsburgh; come talk to me if you’re interested in also working for Disney Research in Pittsburgh. I’m out of time so I’ll be in the hall after this.