cs4fnissue18

ISSN 1754-3657 (Print) ISSN 1754-3665 (Online)Computer Science for Fun Issue 18Machines That Are CreativeThe Sorceror’sApprentice 2.0Can a computertell a good story?Music-makingmates

Can machinesbe creative?In this issue we explore whether machines can becreative. Ada Lovelace suggested in the 1800s thatone day they might, and now computational creativityresearchers are making it happen. We look at the firstattempt at a creative algorithm for writing love letters andmore recent programs that generate novel stories andfunny tweets. Machines also create music: from programsthat evolve better music to ones using it to improve theirrelationships with humans. We even look at programs thatintend to paint portraits and artificial intelligences trying tocreate magic tricks. Whatever kind of art we may want tocreate, the computers are having a go at creating it too.Ada Lovelace: Visionaryby Paul Curzon, Queen Mary University of LondonIt is 1843, Queen Victoria is on the British throne. The industrial revolution has transformed the country.Steam, cogs and iron rule. The first computers won’t be successfully built for a hundred years.Through the noise and grime one woman sees the future. A digital future that is only just being realised.Ada Lovelace is often said to be the first Ada was a mathematician with a creative Ada saw even further though. She combinedprogrammer. She wrote programs for a flair and while Charles had come up with the maths with creativity and so she realised thatdesigned, but yet to be built, computer called innovative idea of the Analytical Engine itself, not only could they store and play music theythe Analytical Engine. She was something he didn’t see beyond his original idea of the could also potentially create it - they couldmuch more important than a programmer, computer as a calculator. Ada saw that they be composers. She foresaw the whole ideathough. She was the first truly visionary could do much more than that. of machines being creative. She wasn’t justperson to see the real potential of computers. the first programmer, she was the first trulyShe saw they would one day be creative. The key innovation behind her idea was creative programmer. that the numbers could stand for more thanCharles Babbage had come up with the just quantities in calculations. They could Ada also wroteidea of the Analytical Engine - how to make represent anything - music for example.a machine that could do calculations so we Today when we talk of things being digital “The Analytical Enginewouldn’t need to do it by hand. It would be - digital music, digital cameras, digital has no pretensions toanother century before his ideas could be television, all we really mean is that a song, originate anything.”realised and the first computer was actually a picture, a film can all be stored as longbuilt. As he tried to get the money and strings of numbers. All we need is to agree So how does that fit with her beliefbuild the computer, he needed someone to a code of what the numbers mean - a note, that computers could be creative?help write the programs to control it - the a colour, a line. Once that is decided we Read on and see if you caninstructions that would tell it how to do can write computer programs to manipulate unscramble the paradox.calculations. That’s where Ada came in. They them, to store them, to transmit them overworked together to try and realise their joint networks. Out of that idea comes the whole ofdream, jointly working out how to program. our digital world.2 www.cs4fn.org

Playing with robotsby Paul Curzon, Queen Mary University of LondonThere are many, many toys to play with these days that contain computers. That means toys are gettingmore intelligent. Toys are starting to be able to play with us rather than just being played with. But why dochildren play? Given how much we do it, play must be important! Many animals play too and often seemto be having as much fun as us. To have evolved, play must increase their chances of survival somehow.Could it be giving a creative edge?Play is partly about practising skills that will Wimbledon final is all-together different from the most creative in coming up with novelbe useful in the future, partly about building having a fun pillow fight. Similarly, when ways to do things. Dolphins naturally play instrength. A kitten that constantly plays at Gareth Bale is bearing down on goal having dozens of ways: from playing with bubblepouncing on things will become a much stormed the length of the pitch, it’s unlikely rings to playing with seaweed. They are alsobetter hunter. An active child will grow up he’s feeling playful. That kind of aggressive, great at coming up with new things to dofitter than a couch potato who never runs competitive play doesn’t seem to be linked and new ways to solve problems set them.around a playground. Is there more to it to creativity. Playful play is though. Whatthan that though? Patrick Bateson of the matters for creativity seems to be a playful What mattersUniversity of Cambridge and Paul Martin mood – feeling positive and light-hearted – for creativity isof King’s College London think it could be more than the activity itself. a playful mood.about creativity too. Researchers have found that the more By playing, humans and animals alike mayWe mean lots of different things when playful kinds of animal, like dolphins, that have more chance to experience situationswe talk about play – playing in your first seem to play for the sake of playing, are also and generate ideas that will help to solve novel problems in the future. They may be exercising their brain as well as their muscles. In the wild, for example, dolphins often use curtains of bubbles to trap fish to eat – just like the ones they play with! Perhaps it was bubble play that gave them the fishing idea. Play does seem to enhance the creativity of children. The jury is still out on whether it leads to adults being more creative too – lots more research is needed to find that out for sure. What is certain is that successful companies don’t just want to employ hard- working intelligent people. They want their workers to be creative too and many high tech companies that rely on innovation, and so the creativity of their staff, already actively try and foster playfulness. Google is perhaps the best example. The company plays with its logo daily and its offices are famous for the playful atmosphere. You do not have to stop playing just because you grow up. People who see work as a place to play may well be the most valuable employees a company has. Now, if play is important in helping both humans and animals be creative, then perhaps we should be creating machines that are playful too. Perhaps the intelligent machines of the future will need to play to achieve their own potential just as much as we do. Robots are likely to be our children’s playmates in the future. Maybe the robots will need playmates as part of their growing up too. [email protected] 3

The algorithm thatcould not speak its nameby Paul Curzon, Queen Mary University of LondonThe first program that was actually creative was probably writtenby Christopher Strachey, a descendent of first programmer,Ada Lovelace, in 1952. It wrote love letters…possibly gay ones.The letters themselves weren’t particularly lists of different kinds of words chosen to It then combined several similar rulesspecial. They wouldn’t make your heart be suitable for love letters. There was a list about different kinds of sentences to give askip a beat if they were written to you, of nouns (like ‘affection’, ‘ardour’, …), a list different love letter every time.though they are kind of quaint. They of adjectives (like ‘tender’, ‘loving’, …), andactually have the feel of someone learning so on. Strachey knew Alan Turing, who was a keyEnglish doing their best but struggling with figure in the creation of the first computers,the right words! It’s the way the algorithm It then just chose words from the and they may have worked on the ideasworks that was special. It would be simple appropriate list at random and plugged them behind the program together. As both wereto write a program that ‘wrote’ love letters into place in template sentences, a bit like gay it is entirely possible that the programthought up and then pre-programmed by slotting pieces into a jigsaw. It only used a was actually written to generate gay lovethe programmer. Strachey’s program could few kinds of sentences as its basic rules letters. Oddly, the one word the programdo much more than that though – it could such as: “You are my <adjective> <noun>”. never uses is the word ‘love’ – a sentimentwrite letters he never envisaged. It did this That rule could generate, for example, that at the time gay people just could notusing a few simple rules that despite their “You are my tender affection.” or “You are openly express. It was a love letter algorithmsimplicity gave it the power to write a vast my loving heart”, substituting in different that just could not speak its name!number of different letters. It was based on combinations of its adjectives and nouns. Diamond Dogs Rock star David Bowie co-wrote a program that generated lyric ideas. It gave him inspiration for some of his most famous songs. It generated sentences at random based on something called the ‘cut-up’ technique: an algorithm for writing lyrics that he was already doing by hand. You take sentences from completely different places, cut them into bits and combine them in new ways. The randomness in the algorithm creates strange combinations of ideas and he would use ones that caught his attention, sometimes building whole songs around the ideas they expressed. An algorithm is the reason his song lyrics are so surreal!4 www.cs4fn.org

Rules of Engagementby Tony Veale, University College DublinWhen you see a child throwing a tantrum on a train, who do you blame: the child, who – though annoying– may not know any better, or the parent, inured to the noise and unwilling to do anything about it?Thinking is something we must all learn to do if we are to do it well, and we must all learn to think sociallyas well as intellectually to successfully engage with the world. The same is true for our machines: toengage successfully with the world, they must engage successfully with us and with each other. They arelike children: when they produce hilariously stupid results, the fault lies as much with us as with them. Ifwe don’t train them to engage with the task at hand, to anticipate the unexpected, and to know when theirrules are about to break down, then we all share in the stupidity that ensues.Bake me a cake as fast the human baker is mostly at fault. Did they this kind of engagement with rules? Theas you can really think this was the cake design? Poor answer is a qualified “yes”: they must be Aunt Elsa. If a person can’t detect this kind programmed in the right way, not just withEven bakers now use technology. Email of problem, what hope for our machines? hard-coded rules, but with knowledge ofthem the picture or text you want on your their own workings, able to reason aboutcake, and they use a special printer (with Breaking Rules their own rules. Building systems like thatfood dyes for ink) to print your design onto is what computational creativity is all about.the icing. It is commonly believed that creativity is Ironically, such programs may have more about breaking rules. If you are going to self-knowledge than a human doing theThis real cake design produced by a New break a rule, start with that one! Nothing same job. By using introspection to designYork bakery is a funny example of what can is further from the truth. Creativity comes them we humans can obtain greater self-go wrong when people and machines don’t from a hyper-understanding of the rules knowledge of how we ourselves work.engage properly with what they are doing. rather than from a willful ignorance of them. We must know the limits of our rules, To find out more about computationalThe customer used Microsoft Outlook to and how to tell the difference between a creativity research check outemail the cake message to the bakers. It convention and a hard rule. People are RobotComix.com which is full ofinserts special HTML tags (code used to creative all the time in chess, poetry and computational creativity comics andformat web pages) into its emails to make soccer without ever breaking the rules: cartoons, including a new book Hand-them look prettier. Unfortunately, the bakery instead, they break with convention! Made By Machines.doesn’t use Outlook, so the HTML was cut- Creativity requires a deep engagement with <b><em>Enjoy!</em></b>and-pasted directly into the cake-printing the rules, to know where individual initiativeprogram. We can laugh at the software but can take over. Can computers ever show [email protected] 5

Yeah, Back in the 1950s, the philosopher Gallie All is not lost. Some of us have beenBut is It? introduced the idea of essentially contested concentrating on the perceptions that people concepts: concepts that lead to endless have of software being creative or not,by Simon Colton, Goldsmiths College arguments that can’t be settled by logic or with specific emphasis on the ‘or not’ part. collecting evidence, ones where everyone It’s easy to dismiss software as not being“Yeah, But is it Art?” How claims the others are using the concept the creative, and people’s reasons tell us howmany times do we hear this? In wrong way. There are things in life that we we can change their views, enabling themmagazines, web forums, Twitter, are meant to disagree about! How interesting to appreciate more what software does, andTV and radio: from Damien is that? And scary, especially for scientists what wonderful things it produces. ThereHirst’s pickled sharks and trying to study such a disputed concept. is no creativity gene or algorithm, only theTracey Emin’s unmade bed, to perceptions of people. Think of a white wall.whether videogames are art. The It’s no wonder that nobody has come close In the day, it’s obviously a white wall, isn’t it?argument is never ending and to defining what creativity is, or to explaining Go back at night with the lights off, and thatoften bitter. Thank goodness for why some people are more creative than white wall will be as black as can be. Thethat! Imagine how much more others: it is an essentially contested concept. change in circumstances has changed theboring life would be if we all We need to disagree about creativity so perception you have of it. The same is truesuddenly agreed. As a society, it can be a driving force for change. But of human qualities, like being funny or beingwe have agreed to disagree about it raises the serious question of how to creative. And the same is true of computerart, and the resulting arguments get software to act in creative ways, and programs. So, if we can make it hard fordrive innovation enhancing to get society to accept machines are people to perceive software as not creative,our world. creative, if we’re not prepared to define then maybe, just maybe, they might one day what creativity is. Most other areas of be prepared to call it ‘creative’. Artificial Intelligence are based on concrete definitions, with increasingly sophisticated One argument is that software is uncreative, software written to perform intelligent mini- because the programmer supplies all miracles safely within those definitions. the intentionality. “You write and run the But, in Computational Creativity research, software”, they say, “it’s you that wants to if we embrace discord and uncertainty create something – so it’s not being creative: about the key concept, how can we you are!” And they are right, but that can proceed scientifically? change. By taking these criticisms, we can write software that addresses the issues and makes it difficult to claim that the program is not creative (for example, see the Painting Fool, on the opposite page, and intentionality). “It’s still not creative”, they say, “because it’s…erm… I mean, it’s not… erm…”. That’s the hope, anyway. Only time will tell.6 www.cs4fn.org

The The Painting Fool painted portraits...but only drawing tool. It took a photo for the portraitPainting if it felt like it. When someone sat to have then made a quick sketch of what it wantedFool their portrait painted, it was reading the to achieve. It then set about simulating the newspaper. If you were unlucky enough to use of pencils, paints and pastels. It madeby Simon Colton, Goldsmiths College ask for a portrait when it had just read lots various choices while painting that it hoped of truly miserable articles, it told you to go would enhance the chances of people usingCan a machine intend to create away, explaining that it didn’t think it was a particular word like “bright”, “colourful”a work of art? We looked at this appropriate to paint a portrait when it was or “crazy” (if in a good mood) or “dull”,during a Paris art exhibition in such a (simulated) bad mood. It would “bleary” or “grey” (if in a bad mood) tocalled ‘You Can’t Know my send you packing with a miserable quote describe the portrait.Mind’ with a program called from a miserable article, and suggest you‘The Painting Fool’ come back later, when it might be in a better At the end, The Painting Fool took a long(www.thepaintingfool.com). mood. In the context of ‘You Can’t Know my hard look at the portrait, to check whetherWe want it to be taken seriously Mind’, we hoped that this would emphasise its picture matched the mood it hoped toas a creative artist in its own its independence. In fact, several times, we portray. Finally it told the sitter how well itright, one day. At the exhibition, desperately wanted it to paint a portrait for thought it had done, and whether this waswe explored whether people some VIP, but it refused and we had to live a “great success”, “miserable failure” orwould think of the software as with that decision. somewhere in-between. It also learned fromintending to do things. its successes and its mistakes, so that it For people who caught the software in a not- became more likely to achieve the required so-terrible mood, The Painting Fool would mood with future portraits. We asked people attempt to paint a portrait which captured whether they thought the software had its mood. If in a good mood – after reading exhibited a little intentionality. Now we ask lots of upbeat articles – it asked the sitter to you: what do you think? smile. It made clear that it was using them as a model, rather than them using it as a And yeah, but. Is it Creativity? [email protected] 7

Can a computer tella good story?by Rafael Pérez y Pérez of the Universidad Autónoma Metropolitana, MéxicoWhat’s your favourite story? Perhaps it’s from a brilliant book you’ve read: a classic like Pride andPrejudice or maybe Twilight, His Dark Materials or a Percy Jackson story? Maybe it’s a creepy tale youheard round a campfire, or a favourite bedtime story from when you were a toddler? Could your favouritestory have actually been written by a machine?Stories are important to people everywhere, are studying how we create them. I use how humans do it. I started with a book bywhatever the culture. They aren’t just for computers to help. Why? Because they Open University Professor Mike Sharples.entertainment though. For millennia, people give a way to model human experiences He suggests it’s a continuous cyclehave used storytelling to pass on their as programs and that includes storytelling. between engagement and reflection. Duringancestral wisdom. Religions use stories to You can’t open up a human’s brain as they engagement a storyteller links sequences ofexplain things like how God created the create a story to see how it’s done. You can actions without thinking too much (a bit likeworld. Aesop used fables to teach moral analyse in detail what happens inside a daydreaming). During reflection they checklessons. Tales can even be used to teach computer while it is generating one, though. what they have written so far, and if neededcomputing! I even wrote a short story about This kind of ‘computational modelling’ gives modify it. In doing so they create rules thata kidnapped princess to help my students a way to explore what is and isn’t going on limit what they can do during future roundsunderstand things like bits. when humans do it. of engagement. According to him, stories emerge from a constant interplay betweenIt’s clear that stories are important for So, how to create a program that writes a engagement and reflection.humans. That’s why scientists like me story? A first step is to look at theories of8 www.cs4fn.org

What dagger, jumped towards Jaguar Knight The judge ofknowledge and attacked Jaguar Knight. Jaguar a good storywould you Knight’s state of mind was very volatileneed to and without thinking about it Jaguar A storyteller must be able towrite a story Knight charged against Trader. In a fast judge their work. Rafael createdabout the movement, Trader wounded Jaguar a system that could evaluatelast football Knight. An intense haemorrhage aroused stories. It’s not easy. What makesWorld Cup? which weakened Jaguar Knight. Trader a good story? It’s questions knew that Jaguar Knight could die like this he’s trying to answer.With this in mind I wrote a program called and that Trader had to do something His program evaluates theMEXICA that generates stories about the about it. Trader went in search of some interestingness, coherence andancient inhabitants of México City (they medical plants and cured Jaguar Knight. novelty of a story. The followingare often wrongly called the Aztecs – their As a result, Jaguar Knight was very is its evaluation of the Jaguarreal name is the Mexicas). MEXICA grateful towards Trader. Jaguar Knight Knight story: do you agree with it?simulates these engagement-reflection was emotionally tied to Trader but If not can you explain why not?cycles. However, to write a program like Jaguar Knight could not accept Trader’sthis you need to solve lots of problems. behaviour. What could Jaguar Knight do? This story is really good! I like it!For instance, what type of knowledge Trader thought that Trader overreacted; Congratulations!does the program need to create a story? so, Trader got angry with Trader. In thisIt’s more complicated than you might way, Trader - after consulting a Shaman Here are some comments about your workthink. What knowledge would you need to - decided to exile Trader. that I hope will be a useful feedback.write a story about the last football WorldCup? You would need facts about As you can see it isn’t able to write CoherenceBrazilian culture, the teams that played, stories as well as a human yet! The The story is very logical; all actions are nicelythe game’s rules… Similarly, to write a way it phrases things is a bit odd, like integrated and form a coherent unit.story about the Mexicas you need to know “Trader got angry with Trader” ratherabout the ancient cultures of México, than “Trader got angry with himself”. It’s At the end most difficulties are solvedtheir religion, their traditions, and so on. missing another area of knowledge: how (although there are few conflicts that youFiguring out the amount and type of to write English naturally! Even so, the should crack earlier). Good!knowledge that a system needs to narratives it produces are interesting andgenerate a story is a key problem a tell us something about what does and Interestingnesscomputer scientist trying to develop a doesn’t make a good story. And that’s The plot starts with some tension.computerised storyteller needs to solve. the point. Programs like MEXICA helpWhatever the story you need to know us better understand the processes and The story reaches a nice climax with a goodsomething about human emotions. knowledge needed to generate novel, amount of tension. This is an importantMEXICA uses its knowledge of them to interesting tales. If one day we create characteristic of a good narrative. Great!keep track of the emotional links between a program that can write stories as wellthe characters using them to decide as the best writers we will know we A better end would contribute to having asensible actions that then might follow. really do understand how humans do more interesting tale. it. Your own favourite story might not be written by a machine, but in the future, There are surprising events that make the you might find your grandchildren’s story appealing. I enjoyed that! favourite ones were! NoveltyBy now you are probably wondering For Rafael’s short story ‘A Godlike I find this story pretty original! I love it!what MEXICA’s stories look like. Here’s Heart’ go to www.cs4fn.org. If youan example: like to write stories, then why not My evaluation of your story is 100/100 learn to program too then you couldJaguar Knight made fun of and laughed try writing a storytelling programat Trader. This situation made Trader yourself. Could you improve onreally angry! Trader thoroughly observed MEXICA?Jaguar Knight. Then, Trader took a [email protected] 9

Dangerousby Pablo Gervás and Carlos León,Universidad Complutense de MadridRussian tales, virtual worlds, dangerous curves...There are lots of sources of inspiration forprograms that write stories! Pablo Gervás andCarlos León tell us about some of the ways theirteam at the Universidad Complutense de Madridare exploring. How long before you are watchinga Hollywood film where the storyline is creditedto a program?The shape of stories to comeOne way to devise an algorithm to write new stories can happen – a villain must have a sword if he is tois to look at the structure of existing stories. People stab a prince, for example. It also knows what thehave studied literature for centuries, and have had consequences of those actions are – that princesome useful ideas about the shape stories take. will be hurt if stabbed! Although Russian folk talesVladimir Propp studied Russian fairy tales and were used as the original inspiration, we’re nowdecided that they all fit a basic pattern: a Hero sets adding more story structures. No doubt you canoff on a journey to undo a wrong perpetrated by think of lots more. The detective who has to solvea Villain, they may be aided by a magical Helper, a crime committed by a murderer hiding in plainand they are rewarded at the end by marrying a sight? The man in the street who saves the EarthPrincess. Sound familiar? Based on Propp’s ideas, from an alien invasion after the authorities fail? Byour team have developed a program called Propper extending Propper, and collecting enough patterns,that can create stories by following this pattern. we hope to build a reasonably good storyteller: oneTo make sure the stories make sense it knows about that can at least match the formulaic plots of yourthe things that must be true before particular actions run‑of‑the‑mill Hollywood movie!10 www.cs4fn.org

CurvesEvery story under the sunStructure isn’t the only way to produce a good story.When you play a video game the story changes based on what you do.Our team started with this idea as a way of creating generates all events and situations that could possiblystories. Our system, STellA, simulates a virtual world, happen within the limits of that world. It’s as though ajust as video games do. In a virtual world lots of things human writer worked out all possible storylines for theircan happen based on the rules of the world: characters story: like Shakespeare writing out every feasible endingeat, sleep, talk, have hobbies, and so on. As the world for Romeo and Juliet. While it isn’t possible for a humanis simulated actions happen and a story unfolds. STellA writer, computers can do it in seconds. Why write oneis a bit different from a normal virtual world though. It story when you can write them all!Don’t forget The curveyour readers! of a storyWhichever way you write stories, it’s Unfortunately just generating lots of storiesimportant to remember they are written to be isn’t good enough. One of the most studiedread by someone. Your stories will work better problems in Computational Creativity is howif when writing them you keep in mind what to decide on the best result; the most creativethe reader is going to be thinking at each work from those generated. STellA has topoint. “Have I already told them that the old tackle this problem too: if you produce lotslady had a gun in her purse?” “If I say this of simulations, you have to decide whichnow, will it give away the ending too soon?” one gives the story you want people to read.Writers think about questions like that over STellA deals with it in two ways. The firstand over as they write. They then rewrite their is to let the human who runs the programdrafts based on the answer, adding, deleting set conditions for the resulting story: theyor modifying things until the worries go away. might require that the hero doesn’t marry theAnother line of research being followed by princess at the end, for example.our team is to model this kind of behavior bywriters so that we can use it to improve our The other way is to use curves! You pick somewhole family of storytelling programs. aspect of the story, like the amount of emotion involved or degree of danger, and represent itOur model captures the way in which the as a number. Suppose you chose danger. Aswriter (or program) working on a story does the versions of the story progress the degreethese things repeatedly: drafting, reading, of danger in each changes in different waysasking questions about the draft, and and so the number representing it goes upredrafting until all their worries have been and down in different ways too. Now thinkresolved. The story is then told or written of that number changing as plotting a graphdown in its final version. Most existing – the story is being modeled as a curve thatstorytelling programs (including ours) goes up and down as the danger changes. Acurrently cover only a small part of the cycle way to chose a story from the many possiblethat human writers follow. Once we extend ones simulated is to pick the one that bestthem to include the missing operations, the matches a given shape of curve. You mightstories they produce will improve massively. for example choose a story curve where the danger is high at the start, drops to nothing,Put all the different approaches together then builds rapidly to the end. That wouldin one program and Hollywood watch out! give a different story to one where its dangerWhat a story that will make! curve was always low. Based on its ‘narrative’ knowledge of the kind of curve that makes a good story, STellA can choose the best stories generated from its multiple world [email protected] 11

Composing fromCompressionby Geraint Wiggins, Queen Mary University of LondonComputers compress files to save space. But it also allows them to create music!Music is special. It’s one of the things, like throughout life), we learn to remember If we do this to the internal structurelanguage, that makes us human, separating pitch in ever more efficient ways, giving of music, there are little repetitionsus from animals. It’s also special as art, our compression algorithms better and everywhere, and the order that they appearbecause it doesn’t exist as an object in the better chances to compress well. And so we is what makes up the music’s structure.world— it depends on human memory. remember music better.“But what about CDs? They’re objects in If we compress music, but then decompressthe world”, you might say and you’d be Our team use compression algorithms to it in a different way, we can get a new pieceright, but the CD is not the music. The understand how music works in the human of music in a similar style or genre. We haveCD contains data files of numbers. Those mind. We have discovered that, when evidence that human composers do that too!numbers are translated by electronics into our programs compress music, they canthe movements in a loudspeaker, to create sometimes predict musical structures, even What our programs are doing is learning tosound waves. Even the sound waves aren’t if neither they nor a human have “heard” create new music. There’s a long way to gomusic! They only become music when a them before. To compress something, you before they produce music you’ll want tohuman hears them, because understanding find large sections of repeated data and dance to—but we’re getting there!music is about noticing repetition, variation replace each with a label saying “this is oneand development in its structure. That’s of those”. It’s like labelling a book with itswhy songs have verses and choruses: so title: if you’ve read Lord of thewe can find a starting point to understand Rings, when I say thetheir structure. In fact, we’re so good at title you know whatunderstanding musical structure, we don’t I mean withouteven notice we’re doing it. What’s more, me tellingmusic affects us emotionally: we get excited the story.(using the same chemicals that get usexcited when we’re in love or ready to fleedanger) when we hear the anthem sectionof a trance track, or recognise the big themereturning at the end of a symphony.Surprisingly, brains seem to understandmusical structure in a way that’s like thealgorithms computer scientists use tocompress data. It’s better to store datacompressed than uncompressed, becauseit takes less storage space. We think that’swhy brains do it too.Sound wavesonly becomemusic when ahuman hearsthem Even more surprisingly, brains also seem www.cs4fn.org to be able to learn the best way to store compressed music data. Computers use bits as their basic storage unit, but we can make groups of bits work like other things (numbers, words, pictures, angry birds...); brains seem to do something similar. For example, pitch (high vs. low notes) in sequence is an important part of music: we build melodies by lining up notes of different pitch one after the other. As we learn to hear music (starting before birth, and continuing12

Music‑makingmates forMortimerby Louis McCallum, Queen Mary University of LondonRobots are cool. Fact. But can they keep you interested for more thana short time? Over months? Years even?Roboticists (thats what we’re called) have Music is emotionally engaging but in a wayfound it hard to keep humans engaged with that doesn’t seem fake or forced. It alsorobots once the novelty wears off. They’re changes constantly as we learn new skillseither too simple and boring, or promise too and try new ideas. Although there have beenmuch and disappoint. So, at Queen Mary many examples of family bands, duettingUniversity of London we’ve built a robot couples, and band members who werecalled Mortimer that can not only play the definitely not friends, we think there are lotsdrums, but also listen to humans play the of similarities between our relationships withpiano and jam along. He can talk (a bit) people we play music with and ‘voluntaryand smile too. We hope people will build non-kin social relationships’ (as robotocistslong term relationships with him through call them - ‘friendships’ to most people!).the power of music. In fact, we have found that people get the same confidence boosting reassuranceRobots have been part of our lives for a and guidance from friends as they do fromlong time, but we rarely see them. They’ve people they play music with.been building our cars and assemblingcircuit boards in factories, not dealing with So, even if we are engaged with a machine,humans directly. Designing robots to have is it enough? People might spend lots ofsocial interactions is a completely different time playing with a guitar or drum machinechallenge that involves engineering and but is this a social relationship? We testedartificial intelligence, but also psychology and whether people would treat Mortimercognitive science. Should a robot be polite? differently if it was presented as a robot youHow long and accurate should a robot’s could socially interact with or simply as amemory be? What type of voice should it have clever music machine. We found peopleand how near should it get to you? played for longer uninterrupted and stopped the robot whilst it was playing less oftenIt turns out that making a robot interact if they thought you could socially interactlike a human is tricky, even the slightest with it. They also spent more time lookingerrors make people feel weird. Just getting at the robot when not playing and less timea robot to speak naturally and understand looking at the piano when playing. We thinkwhat we’re saying is far from easy. And if we this shows they were not only engaged withcould, would we get bored of them asking playing music together but also treatingthe same questions every day? him in a social manner, rather than just as a machine. In fact, just because he hadWould we a face, people talked to Mortimer evenbelieve their though they’d been told he couldn’t hear orconcern if they understand them!asked how wewere feeling? So, if you want to start a relationship with a creative robot, perhaps you should learn to play an instrument! [email protected] 13

Music thathas sexby Paul Curzon, Queen Mary University of LondonMusic is a creative part of our culture but what drives thatcreativity? Music changes over time as new sounds are inventedand old ones fall from favour. Where does the creativity come fromthough? Is it from the ‘creative people’: the musicians? Or is it youand me that really drive the creative changes, just by the music wechoose to listen to? Could a machine make creative new music welove without an actual composer? Researchers based in Londonand Japan teamed up to investigate. They did it by creating aDarwinian music engine – a musical machine that has sex!Well ok. It doesn’t really have sex, but survive to have babies of their own and In the software version it can be anythingit does do a software equivalent. Life so pass their DNA on. That DNA contains that captures your idea of good and bad.evolved through natural selection and sex information about what made themis central to that. When we have a baby, special enough to do so well, increasing The researchers investigating musicalwe are mixing up the genes of the parents the chances that the baby does well too. culture, did this: their ‘creatures’ weregiving that child qualities from both. Your bits of sound. Their artificial ‘DNA’ – theDNA is a complex code storing all the You can model this in a computer. strings of numbers – represented differentinformation about how to make you. DNA First create a code to represent the music. Each encoded a computerdetermines the colour of your eyes, how properties of the thing you want to evolve program. When that program is executedmany legs you have, even whether you – its ‘genome’. It can just be a string of it plays a short, seamlessly loopinghave a shell or not! It is unique to you – numbers, one for each property. Next, sound sequence. The genome/programno living thing is exactly the same as you create a random set of these strings to determines things like where notes areso (twins aside), nothing else has exactly start things off. Then the sex begins. Pairs placed and instrumentation, thoughthe same DNA to describe them. Sex is of strings are split in half and joined back other things like the tempo are identicaljust a way to split two creatures’ DNA in together. Extra random changes to the for every loop. Sexual combination andhalf and pair up the two halves to make strings add mutation to the process. Test mutation mimic the fusion of existingnew DNA and so a new creature. the newly created population, keeping the musical motifs, rhythms, and harmonies, best and destroying the worst. For that and the invention of novel ones.Natural selection then works because the you need a ‘fitness’ function to decidefittest creatures, most able to survive the what is good and bad. In nature the Their music engine, ‘DarwinTunes’,conditions they find themselves born into, fitness function is your ability not to die. started with a group of short audio loops14 www.cs4fn.org

Breeding art Evolution has been used to create art by a Japanese team. In their system paintings breed by doing things like chopping two pictures in half then re-pairing the halves. The test to see if a picture survives is based on preferences set at the start by a person indicating the style of art they like. The program then runs for thousands of generations judging the results against the preferences, keeping those that fit best for the next round of breeding. The system is based on the observation that paintings by human artists follow a similar pattern of using features of existing art in new ways rather than completely inventing new ideas. The researchers think you could set up the preferences to match the style of any artist creating new ‘Rembrandts’ or ‘Monets’ by starting with their existing paintings. Would the result be a new painting by that artist? They think so!that played random noise. These loops The loops of random sounds quickly evolution got to a certain point, favouredpaired up, sexually reproduced and evolved into music. This was partly but complicated innovations were beingmutated, creating new loops of music. because pleasing chords and rhythms lost, so improvements could no longerThey were left to evolve over 2,500 used in western music started to evolve. build well on those that came before.generations of musical ‘creatures’ with Later, however, the amount of evolutiondaughters replacing their parents on each slowed and there was little improvement The main aim of the experiment wasround. The twist was that the selection after 600 odd generations. This pattern of to understand how musical culturewas based on the likes and dislikes of fast then slow evolution is actually seen develops. It also shows though that bythousands of people who rated the music in the real world: in the wild, the fossil using a Darwinian process machines canclips for how much they liked them. record and in lab experiments. To work make pleasing music without a composer.Only the top 100 survived in any round. out what was going on with the music the However there is more to it than just team carried out other experiments using natural selection – to be as creative Their methods devised by biologists studying as human composers driving musical ‘creatures’ the evolution of bacteria. It turned out cultural change something else is needed. were bits of the slow-down was mostly because of It is more than just sex and mutation! sound a decrease in the accuracy of how the Creative machines will need some other music was transmitted. A similar thing spark that composers have. arises in early musical cultures when, as musicians learn existing complex musical themes they make mistakes so the original themes are lost. Once the [email protected] 15

The Sorceror’sApprentice 2.0by Howard Williams, Queen Mary University of LondonA good magic trick makes you feellike you’ve just witnessed a miracle.An impossible event has occurred rightin front of you. Most magical effects aresimple to experience, but a fiendishcomplexity lies behind the scenes thatis hard to work out. The methodsbehind some of the best tricks areso complex no-one would everbelieve the magician would goto such enormous lengths topull off such a simple seemingtrick. Magicians will doalmost anything to makeyou gasp in wonder, andgo to similar lengths toprevent you finding outhow they did it!The ultimate sorcerer’s The sorcerer’s apprentice is fed with lotsapprentice: a computer program of information about how we perceive thethat suggests new magic tricks world. Based on that information it churns out new magical methods, leading to newMagicians constantly look for new ways to With a little help from in-the-know tricks, that should amaze an audience inwow an audience. They often head the queue magicians, who have revealed some of their the best way possible. It’s a program that isto try out new technology, and often invent arcane secret methods, our research team able to find the very best version of a trickthings to use in tricks that go on to be used have created what could one day be the at the click of a button!elsewhere. Magicians were involved in the ultimate sorcerer’s apprentice: a computerbirth of cinema, exploiting and refining the program that suggests new magic tricks. Amongst the recent tricks it has comeways film could be edited and manipulated to up with are a magical jigsaw puzzle, andcreate magical movies. Today’s blockbuster Computers are far better than humans at an astounding card trick during whichspecial effects are the end product of this on- doing sums incredibly quickly, and storing a mobile phone reads the mind of ascreen conjuring. huge amounts of information without spectator. All the cunning of a magician’s ever forgetting it. As a result, they are mind is needed to know what will fool realAt Queen Mary University of London we’ve exceptionally good at picking out patterns people – but it takes a clever AI to figurebeen turning our coding skills towards where, to the human brain, there don’t out the ultimate way to really confoundmaking new magic tricks using Artificial seem to be any. They can direct all their them. When performed well, the tricksIntelligence techniques from our labs. number crunching abilities at complex really do leave the spectator thinkingArtificial Intelligence (AI to us geeks!) isn’t problems that would otherwise take humans a miracle has occurred. We know thisjust a Steven Spielberg movie, but a whole years or even centuries to complete. For because, being scientists, we tested thefield of Computer Science. It’s dedicated to example, computers are currently used in tricks out to see how mystifying they were!finding ways to make computers intelligent: to medicine to help scientists understand how Result: very!program computers to give them ever more DNA works. They sift through the billions ofsophisticated ways to solve problems and ways in which different bits of DNA interact We may prefer to have an actual persondiscover new information from huge amounts to cause all sorts of changes in human perform the magic for us (though robotof data. Soon, artificial intelligences will be bodies. These same pattern finding abilities magicians are just around the corner too),driving cars on our streets (they already fly of AIs can be used to sift through the but what magician wouldn’t want a handyplanes in our skies!), cleaning our houses, various ways to build magic tricks and find AI assistant around to help them craft theirrunning our home’s heating systems, possibly the ones that work the best. next masterpiece? The audience needeven being our friends or work colleagues! never know!16 www.cs4fn.org

The rise andfall of theliving dead!by Peter W. McOwan, Queen Mary University of LondonZombies seem to be everywhere these days and if playful fightingagainst the undead is your hobby, there is an app for that. Createdby Queen Mary University of London student David Kilanowski,Trapdoor Zombies adds creative twists to the classic zombie chaseplus some floor dropping gameplay for good measure. There is a twistin the way the game was created - a creative game-playing ArtificialIntelligence (AI) helped.The artificial intelligence program creates endless waves of (not too intelligent) zombiesmoving through level maps of increasing difficulty. The cunning layout of each ismachine‑designed and tested to ensure it is playable. Peppered throughout the levels aretrapdoors. If your thump your phone, the motion sensor opens the trapdoors and dropsanything unlucky enough to be on top to their doom below, be it zombies or you. This fastpaced, fun strategy game sees you race against the clock to shoot the spawning hordes,drop the zombies through the trapdoors and release in-game characters to build your zombiebusting team. To make sure the AI was producing usable designs David experimented withthe many variables that control the gameplay and difficulty, testing each till he found the rightmix. Once they were in the code the machine was ready to create.If you fancy taking on the undead or just seeing how AIs can design games, download theapp at http://www.qappsonline.com/apps/trapdoorzombies/ Magic through the trapdoor You take a deck of cards, shuffled by a spectator, and deal them into two piles. Three cards are selected from one pile, and placed face up. Various face down cards are dealt on to these face up cards. You use the values and suits of the three face up cards to correctly predict a card and its position in the face down pile. If your spectator tries to repeat your amazing feat they fail. It’s a trick that can’t be explained! FInd out more at [email protected] 17

Tours of Babelby Tony Veale of University College Dublin When you wish big, be careful what you wish for. Take a story called The Library of Babel by Argentine writer Jorge Luis Borges. The library of the story is a fantastically vast construction of hexagonal rooms, containing every book that was ever written,as well as every book that can, or will, ever be written.The library is finite but huge: it contains every bookthat can be written with a 25 character alphabet in410 pages at 80 characters per line and 40 lines perpage. It’s easy to imagine such a library, and easy inprinciple to construct one too, but the library is not asuseful as it might seem. For every good book hiding onthe shelves, there are countless millions of nonsensebooks containing random text, and worse, countlessmillions of half-nonsense books that mix real insightwith random nonsense. There is no definitive cataloguefor the library, but since a catalogue is itself just a book,the library contains millions of books claiming to becatalogues, almost all of which are gibberish, whilemany are just misleading.Borges showed us that extremes are easy: to be seen as creative by an outside person, pastiche, because these two extremes aregenerate everything we just need an alphabet the generator itself can’t tell the good the easiest to implement. They also need theand a simple way to combine them, and to outputs from the nonsense, just like Borges’ least amount of knowledge about the worldgenerate nothing we need just do nothing. library has no way of separating the good to be built in to the software.The real challenge is to generate something books from the bad. What if we start within between with meaning and literary merit an existing book by a respected author, To see mere generation and pastiche in thethat is not random and not a rehash of what and make scripted changes to this book wild, just go to Twitter: some of the biggestsomeone else has already written. to obtain a new one that is different but offenders are human, relying on the same old meaningful? We call this pastiche if the new tropes and cut’n’paste techniques, but thereSimple ways combining letters or methods text remains close enough to the original are plenty of computers too, in the guise ofthat cut up existing texts and put them to successfully piggyback on its meaning. Twitterbots. Some of these ‘bots use mereback together in random orders are called When we build software systems to be generation and pastiche to humorous effect,mere generation, generation for the sake creative like humans, it is tempting to build such as @pentametron, which puts togetherof it. Though the results might occasionally ones that specialize in mere generation or in pairs of random tweets from real people if18 www.cs4fn.org

Createthey can be given a poetic cast in iambic apply their own filters about meaning and combinations of words that fit a template likepentameter. Here’s an example: “Still waiting beauty to figure out what they are trying “An X is a Y”, but how would you choosefor the good in that goodbye” & “It’s really fun to say and to determine how well they are X and Y to offer a valuable perspective thatforgetting to reply.” The trick is that the reader saying it, throwing away the worst and only can help people to think about a familiaris willing to imbue this pair of lines with the tweeting the best. A good example of this topic in apt new ways? If you have someconnective tissue of real meaning, even if the new kind of bot is @MetaphorMagnet, which good ideas, why not put them into actionlines are only chosen for their obvious rhyme. generates a novel but meaningful metaphor with a Twitterbot of your own? every hour or so. Here is an example: “ToComputational Creativity researchers are some amnesiacs, memory is a treasured One day the majority of users of Twitter maynow building Twitterbots that use their blessing. To others, it is an overlooked be bots, bouncing texts and ideas off eachknowledge of the world to generate micro- error. #MemoryOrBlessing” How might you other in a magnificent society of artificialtexts that the bots themselves consider build a metaphor machine that generates minds, as we humans look on in wondermeaningful and worth sending out into the new but meaningful metaphors? It’s easy and amusement.world. This new generation of bots must to build a machine that generates random [email protected] 19

Back (page) on the blockby Peter W. McOwan, Queen Mary University of London Blocks, acting as a verb, to block, get in the way (of creativity). As a noun, a block, they are things we use to create stuff. That’s the thing about blocks, you’re never quite sure what you’re going to get. Here we take a Computer Science look at the ways this block bimodality behaves.Block on the doing in Amsterdam! Building with bricks is giving us a better understanding of how welandscape so yesterday! In China waste materials are compute, or don’t, in our heads. recycled to create blocks to print a cheapFancy visiting Denmark? You can without bungalow in a day or so. Others are thinking Big benefit: Broken blocks of brain beckonleaving the comfort of your computer thanks cuddly, looking at how modified 3D printers mental blocksto the Danish government. They uploaded can join together blocks of knitting fora 1:1 map of Denmark into Minecraft, making soft toys. Block bends beatwhere every Danish detail is recreated by behaviourmillions of tiny computer graphics blocks. Big benefit: Blocks are bricking it andRather than millions of hours of game play building buddies Cities like New York are built on a grid.to build the landscape, the full sized replica Streets run in parallel creating city blocks,was created using geographical map data. Block calculations which make it easier to get around. ButThe upload was a hefty Terabyte of block not all cities are so simple to navigate andbuilding data – a billion or so blocks of The brain, like a computer, is a powerful that’s where maps come in. Where we1s and 0s. The Danish government hope information processor. To do the are changes the things we see and feel,players will enjoy wandering the land, monumental calculations needed to stay but what if it could also change the musicadding new buildings and places. The only alive, a brain has different blocks for we’re listening to? Enter Geosound, anrules: no swearing, no bullying no use of different abilities. Your eyeballs are the only app that uses the map of where you arevirtual TNT! part of your brain you can touch. They are to resequence the music on your phone. at the front of the brain but the process Select a track and the app calculates itsBig benefit: Big bulks of blocks bring bi- of seeing starts to happen at the back. beat structure, and then downloads a maplocation benefits Information from your eyes runs through of the streets around you. It extracts the brain areas called, simply, V1 to V5. Damage roads and junctions of your location andBuild-a-block area V3 and you can’t perceive colour. uses them to jump the music through its Damage V5 and you may not be able to see beat structure while showing how the beatArchitects are taking blocks to a whole new movement. Your ability to produce fluent changing elements are moving around youlevel. 3D printing is used to build prototypes conversation seems to happen at the side in on the map. If your location gives an excitingof designs. The printer moves, controlled by a place called Broca's area: problems there rework of a musical classic you can tweet ita computer, depositing blobs of materials cause difficulties with speech. Dyscalculia, for others to experience. Download it from:that stick together in layers, building up the a rare condition where the patient can’t do www.qappsonline.com/apps/geosound/desired shape. But think bigger. What if the arithmetic also seems to have particularprinter was the size of a cargo container brain areas associated with it. We can Big benefit: Blocks and beats makeand deposited blocks of material to create explore the brain’s processing of information mash‑up musica full sized house? That’s what they are in these areas using clever algorithms,cs4fn is edited by Paul Curzon, Jo Brodie and Peter W. McOwan of Queen Mary University of London. Ursula Martin, Bernard Sufrin and Peter Millican of Oxford University gavevaluable input. This issue and/or the research in it has been supported by the projects: CONACYT-México, project number 181561, WHIM (611560), Lrn2Cr8 (610859), ConCreTe(611733), PROSECCO (600653) and COINVENT (611553) funded by the European Commission, Framework Program 7, the ICT theme, and the Future and Emerging TechnologiesFET program. It has also been supported by the Department of Education, Mayor of London through Teaching London Computing and Google. Autumn 2014. www.cs4fn.org