Javascript: The Good Parts, adding icons for the Reactickles and fixing bugs in KeyboardFountain

After reading Javascript: The Good Parts by Douglas Crockford I decided to make some minor changes to the codebase of the suite:

  1. Instead of using the postfix increment operator ++ use += 1
  2. Instead of using the == comparison operator which uses type coercion , use the more reliable ===. Similarly for !=, use !==.

After making those changes, I screen grabbed each Reactickle to make an icon. Midway through the process I realised that it was difficult to tell the difference between some of the mouse based interactions and the keyboard interactions, so decided to make animated video icons instead of a still.

I used Screeny to be able to grab at a fixed pixel dimension of 256×256, but drawn at 128×128 in the main menu.

After adding the video thumbnails I spotted a bug with the KeyboardFountain – objects were being added to the matter.js simulation, but never removed.

First I made sure that the array holding all the circles was empty before I started adding to it:

 while(this.circles.length > 0) { // https://stackoverflow.com/questions/1232040/how-do-i-empty-an-array-in-javascript
 this.circles.pop();
 }

Secondly, I made sure the matter.js World had been cleared before adding new elements to it:

 World.clear(world);

Finally, I noticed that the circles in KeyboardFountain were being drawn incorrectly, overlapping with each other. I realised two things:

  1. I needed to set the ellipseMode to RADIUS:
    ellipseMode(RADIUS);
  2. I needed to draw the circles in the simulation in a new way because of this mode change:
    ellipse(0, 0, r); //previously ellipse(0, 0, r *2)

    The updated suite can be seen here.

Fixing bugs in the beta version of the suite

After making the first version of the full suite, I duplicated the project and set to work on ironing out bugs.

I added the title of the Reactickle that is currently running to the back button which allows you to return to the menu screen.

 this.updateButtonText = function(textToAdd){
 this.title = textToAdd+"\nClick to return to Main Menu";
 }

I used the “\n” special character to make sure that the text instructions were on the next line.

While looking at the discussion in the above link, I discovered that the text() command in p5.js has optional width and height parameters, to make sure that the text stays within a defined area. I implemented this throughout the code.

 this.draw = function(){
 textSize(characterSize);
 noStroke();
 fill(this.backgroundColour);
 rect(this.position.x, this.position.y, this.dimensions.x, this.dimensions.y);
 fill(this.textColour);
 text(this.title, this.position.x+5, this.position.y+20, this.dimensions.x-5, this.dimensions.y-20); //https://p5js.org/reference/#/p5/text
 }

Taking each Reactickle one by one, I started with KeyboardScalingCircleGrid.

On line 14 of KeyboardScalingCircleGrid.js, I had a console output, detailing the size of the array that held each of the ScalingCircles, one for each keyboard button I was taking input from. Repeatedly starting the Reactickle and going back to the main menu resulting in the following output:

KeyboardScalingCircleGrid.js:14 The size of the circles array is 216
KeyboardScalingCircleGrid.js:14 The size of the circles array is 252
KeyboardScalingCircleGrid.js:14 The size of the circles array is 288

It was clear that I needed to treat the array in a different way, by emptying it at the beginning of the setup method of KeyboardScalingCircleGrid.js. Looking in the Javascript basics part of the p5.js wiki pointed me at the MDN array reference page, which didn’t have a clear method. I found a StackOverflow post on how to empty an array in Javascript, and ended up using the following code:

while(A.length > 0) {
 A.pop();
}

I added this to every other Reactickle that had a similar array bug.

While testing every Reactickle to confirm that I’d expunged the array bug, I discovered that that both KeyboardSquares.js and KeyboardFountain.js used a centred rectMode() rather than a cornered one. I added a rectMode() change before each of their draw calls, as well as reseting to corner mode in the main sketch.js to ensure that the GUI was drawn correctly.

Next will be creating some icons for the Reactickles, rather than the current placeholder text.

 

Session 10: More Crystal Growing, Digestion as Computation and Speech Synthesis

We began the tutorial with Jayson’s homework from the last session:

  1. Continue to experiment with Experiment with LEDs in the crystal growth structures and computer case – wax protection? Clingfilm?
  2. Send dissertation proposal and bibliography
  3. Send over the narrative of the space
  4. Speak to Theo about your concerns about computation relevancy

Since the last tutorial, Jayson sent the following images of his experiments:

He’s been experimenting with different chemicals: copper sulphate, magnesium sulphate and aluminium sulphate. Magnesium sulphate aka epsom salt tends to grow in a fairly “clumpy” fashion, see his mask experiment from the previous tutorial. Copper sulphate is is preferred choice. He feels like he’s got to a good level of knowledge now of what works and doesn’t. Both vaseline and cling film didn’t work as a method of keeping the crystals away from certain area of his sculptural objects, as the crystal solution needs to be hot initially.

Dianne suggested trying to use latex after Jayson’s further experiments with taping failed. He has a variety of scalpels that he’s been working with, but it’s a very delicate operation. Another problem he has encountered is that most of his objects float – he’s been having to use a combination of plastic forks and tape to hold things in the baths of solution that he has been preparing.

Jayson has purchased a container of de-ionised water to prepare his solutions with – the first he ordered sprang a leak in the post unfortunately. Tap water has lots of calcium in it which ruins the growth of other crystals. His next growing attempt will be using the PC case that he showed in the previous session. He’s going to document the process thoroughly.

Jayson spoke to Theo in general about his work, but I suggested that he needs to be specific about his final project and his concerns about computational relevancy.

In between sessions I found the upcoming event featuring Yesenia Thibualt-Picazo’s work on Craft in the Anthropocene. Jayson responded by stating that he isn’t so concerned about materiality – he’s keen for the crystals to appear mysterious and alien. He wants the exhibition to make people feel like they witnessing an intelligence of some kind emerge – that super saturation of information gives birth to crystals, in the same way that super saturation of chemicals does.

Jayson sent me his draft essay on Crystal Media Archeology, which has the following opening paragraph:

This essay discusses the process of crystal growth in relation to the development of computers, technology and network culture and how the process of crystallisation is inherent across all kinds of complex systems. Using a hybrid approach of looking at these developments through the methodologies of Variantology (Zielinski, 2006) and Media Archaeology (Parikka 2010), I will discuss a different way of looking at the history and evolution of computers and computational systems. I propose that computers and computational systems are a type of crystallisation that has continued throughout history, this global crystallisation process started with the coalescing of atoms and particles to create complex forms of matter in deep time has continued throughout the history of the universe and is apparent now in the rapid growth and increased density of technology and data.

He also sent me his notes on his Crystal Punk mythology:

In the early part of the 21st century the endless digitisation and quantification of information led to a supersaturation of data. At this point in history there was around 44 zettabytes of data which was stored on enormous data centres and cloud computing servers that stretched all over the world. In the same way that the supersaturation of matter in a liquid causes the crystallisation of that matter the huge amount of density of information caused the crystallisation of data into physical forms. What began in a cloud server farm in china and spread throughout the infrastructure of the cloud and all its connected technology, in the same way a computer virus spreads throughout the internet this crystallisation spread throughout the internet and crystals emerged on cloud servers, hard drives, personal computers, laptops and even memory sticks throughout the world.

Both are shaping up well. He’s going to go on a trip to China soon and was keen to gather more photographic content there – his narrative is that Chinese server farms is where these new crystal first emerge.

We moved on to discuss what he is planning to exhibit in his show:

  1. A screen playing a video of an explanation of his crystal punk mythology – i.e. crystals have emerged, why?
  2. A series of artefacts with crystals growing from them – a phone, a computer and USB memory sticks. I asked how they would be presented? Jayson said that he was going to research vitrines to protect the delicate crystal forms.
  3. A display of crystalmancy tools – drawing on the Hopi practise of using crystals to see the future. He demonstrated a text based game of life simulation, I suggested that this might be a good way of showing how people start to incorporate these crystals into their every day life and customs.

I set the following homework:

  1. Re-approach Theo about computational relevancy
  2. Continue with crystal growing around the computer case
  3. Get kit list worked out for show and consider the layout
  4. Ask the technician team about the availability of vitrines and plinths to display objects within
  5. Do some investigations about how light is cast through his crystals
  6. Continue to work on text based Game of Life simulation and how it can be incorporated into his show
  7. Look at museum displays for ideas on how to label his project and individual pieces

We then moved on to Dianne’s work, beginning with her homework from the previous previous tutorial:

  1. Continue to work on dissertation with Helen Pritchard
  2. Research Markov Chain’s specifically for use within her exhibition
  3. Send me her proposal for her dissertation and the bibliography for it

Since the last but one tutorial, Dianne sent me her dissertation proposal, which has the following title:

HUMANn2: Making Kin with the Aliens Inside

or

Disembodied Data: Decoding our Inner Ecologies

with the following description:

Using the human micro-biome as a lens I aim to question our understanding, coding and decoding of the complexity of life be it computational or biological, questioning what are the definitions of life that would apply to the human or a digital entity. Investigating the ever-expanding organic digital interface, referencing DNA data storage, slime mold logic circuits, synthetic biology, etc.

Dianne related her research on Hidden Markov Model‘s (HMM’s) – that they are used within bioinformatics to attempt to extract gene and pathway information from huge amounts of short DNA/RNA chains, which is what is produced from an attempt to sequence a metagenome (genetic material recovered from environmental samples). Researchers are attempting to make in-silico DNA to make a model of how the human gut works based on this research. All of her research on this research is going into her dissertation. Her proposition for the show is to re-imagine contemporary science practise (specifically molecular biology) around the micro-biome.

We discussed that aliens from another world might see humans as mere carriers or husks for our gut biome. Dianne referenced the ENCODE (Encyclopaedia of DNA Elements) project, and pointed out that they are using computational systems to understand life, which means that our explanations for life will be computationally based. Just like computers, genes have loops within loops, subroutines all within a bigger operating system (the body). Life is computation and computation is life. This brought up a discussion about her previous research into slime computing and the like.

We then moved onto a discussion of what her show would comprise of:

  1. A biographical introduction – starting with her mothers work as a microbiologist.
  2. A biological introduction to the gut biome
  3. Explanation of computational techniques for understanding the biome
  4. Explanation around computational aspects of DNA
  5. Exploration of the future of computation – that it will be biological. Could this comprise of artefacts? Might people eat something to compute it in the future and then poo out the results? Dianne has an idea about using silicon tubes to make a sculpture, but also echo’ing the look and feel of scientific conference posters – she’s recently made one for an art show she’s curating. I referenced the bone gun from Existenz:

I set the following homework:

  1. Think about the narrative of the show – what does she need to make?
  2. Send the poster for the show she is curating
  3. What objects or scenarios are you going to explore in the show? Inpill/outpoo? Could people encode information in DNA as a secure backup? What happens when people die with this information inside them? What happens when they kiss or have sex? Could these be future secure methods of data exchange?
  4. Continue working on dissertation
  5. Create diagrams for explanations using p5.js.

We then went onto Jules’ homework:

  1. Get going in speech
  2. Send me dissertation proposal and bibliography, as well as project proposal
  3. Get going in matter.js (less important)
  4. Keep chasing the technical team for projector resolution and borrow Android tablet
  5. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  6. Make a kit list for the exhibition

Previous to the tutorial, Jules and I spent some time refactoring her code to allow her to track down why her speech synthesis work with p5.speech wasn’t functioning. The refactor was successful, and by the end of the tutorial proper we had her software reading out parts of the conversations between people that she’s collected up to now.

We then discussed her dissertation proposal, entitled:

The Age of Technological Love

She’s going to start writing the dissertation in earnest immediately. We decided that working with matter.js wasn’t necessary for this project and so re-prioritised work on getting swipe interaction working with the Android tablet she’s been given by the technician team. She’s going to get tablet after this tutorial and begin working with it. She’s also made progress on the projector selection, but hasn’t finalised it yet. We went through a possible kit list for her exhibition:

  • Projector
  • Plinth for tablet
  • Mount for tablet
  • Computer for projection
  • Speakers
  • Signal interconnect for all devices
  • Power for all devices

I reminded her not to forget any necessary adaptors – especially for projector/computer connection.

I set the following homework:

  1. Continue working on speech interaction.
  2. Firm up projector and tablet models.
  3. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  4. Finalise kit list.

Making the Fountain part of the physics simulation and creating a suite of Reactickles

After trying and failing to get a triangle to represent the fountain in KeyboardFountain, I reverted to using a rectangle, which functions nicely.

The new sketch can be seen here.

After talking to Wendy, we agreed that the next step was to get all the individual sketches working in a suite.

I created a new folder to contain the suite: Reactickles3, and started by trying to run KeyboardScalingCircleGrid within a larger p5.js sketch.

I created a new sketch.js in the Reactickles3 folder and copied KeyboardScalingCircleGrid.js into the same folder. After making the entire KeyboardScalingCircleGrid a class of its own, I was able to call its setup, draw and keyTyped methods from within my main sketch.js.

With that functioning, I started to test other Reactickles in this new structure, starting with KeyboardWorm, followed by MouseWorm, KeyboardSpringyCircles, MouseSpringyCircles, KeyboardSquares and finally KeyboardFountain.

Once those were working, I created an array of those Reactickles as well as a main menu screen and a return to main menu button. There are still some bugs in the system, but I’m going to save those for the next session.

Session 9: More Markov Chains, Speech Synthesis and Crystal Punk

Unfortunately, Diane was unable to make the session, but she did send updates on what she’s been working on recently. She’s been concentrating on writing her dissertation but also finding out more information about Markov Chains. She’s also been looking into Rebecca Febrink‘s Wekinator project. She’s hoping to use some of the data sets from the Human Microbiome project and using those to train Wekinator to turn out endless streams of Biome sequences – so that then never repeat. She’s going to explore possible outputs and bring them to the next tutorial.

Jayson, Jules and I had a discussion about how planning for the end of year show is going. Jayson went to a show discussion meeting with Atau Tanaka. This year they have both ground first floors of the exhibition space, but positioning within the spaces will be decided at the next meeting. Jules is keen to have a large dark space to show her projection. Jayson described how last year everyone drew onto a large floor plan diagram of the space, and then revisions were made. I suggested that it would be great if they could all be close to each other – that this would be better than all having disparate separate rooms.

As Jules was absent from the last tutorial, we started with her homework from the previous tutorial:

  1. Gather more content, dependent on Tumblr responses.
  2. Make a prototype with:
    1. p5.js
    2. Data files in json for conversations
    3. Physics based chains of conversations, similar to this Matter.js example.
    4. Voice synthesis with p5.js-speech.
    5. iPad swipes via osc using HammerJS and p5js-osc.
  3. Keep chasing the technical team for projector resolution
  4. Set up your own GitHub.

She’s still trying to get more content from other people, but is happy with working with her own content for now. We went through the setup of Git, Github source hosting and Github webpage hosting via Github pages. I pointed her in the direction of various tutorials on Github setup and Git in general. She’s successfully pushed to GitHub.

We took a look at saving her text content in external JSON files, but realised it was faster for her to save the data directly in the source code of her p5.js project for now.

Jules is still going through the setup of Matter.js and p5.js-speech. She’s been in Japan recently and noted that they have a very different form of online intimacy there – it feels faster there, whereas in the West people build slower. She found people using Tinder and Grindr depending on their preference, but the Pairs dating app was also popular.

In terms of her dissertation she’s been reading “Control and Freedom” by Wendy Chun again, which was her original inspiration for this project. She’s fascinated by the idea of aura from Walter Benjamin’s The Work of Art in the Age of Mechanical Reproduction. She stated that Tinder or other social dating networks diminish the aura of relationships – they aren’t your waking, walking life. The only way to get to aura is by sitting down with people in the real world and communicating with them.

Moving on to the coding of her project, I suggested prioritising speech synthesis first, then physics simulation and then finally interaction with a tablet computer to allow for the familiar swipe gesture to be used to navigate her curated dataset.

I suggested drawing out all the text to an image and then redrawing that every frame, rather than laboriously redrawing the entire ellipse of text on every frame. We went through the p5.js-speech library, which seemed straightforward, although she did follow up with me later stating that she’d been having some problems. We are going to have a separate coding only meeting before the next tutorial.

Finally we moved on to securing the Android tablet computer that she’s going to be leant for the exhibition. We found a suitable device to secure it. Finally we discussed how much explanation is going to be visible at the exhibition, something which still needs to be resolved. Is she going to be there to talk it through with visitors for the duration?

I set the following homework:

  1. Get going in speech
  2. Send me dissertation proposal and bibliography, as well as project proposal
  3. Get going in matter.js (less important)
  4. Keep chasing the technical team for projector resolution and borrow Android tablet
  5. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  6. Make a kit list for the exhibition

We then moved onto Jayson’s homework from the previous tutorial:

  1. Experiment with lasers, LEDs and other electronics in the crystal growth structures.
  2. Read the Curious Lore Of Precious Stones.
  3. Think more about the overall look of the exhibition.
  4. Send me his proposal for her dissertation and the bibliography for it

In between this tutorial and the previous one I had sent Jayson an article from the Washington Post on how human activity has caused a whole new layer of minerals to form around the world.

Jayson showed some of his experiments with trying to grow crystals around electronics components:

I suggested covering the contacts of the electronics with wax and then removing it using a hair dryer after the crystals have grown. He’s going to continue experimenting.

He ran out of copper sulphate, so he bought a 25 kg bag! It seems that it’s used in the sheep farming business. He’s also been thinking about covering an entire PC case in crystal, so has also bought one of those:

He’s also been experimenting with growing crystals on lasers, but has been having similar problems to those that occurred around LED’s. I suggested trying wrapping the components in clingfilm and then carefully removing afterwards if the previously suggested wax technique was ineffective.

We moved on to his thoughts on “The Curious Lore of Precious Stones“. Jayson has been steadily working his way through the 500 odd pages of fairly dense writings on crystal lore. He discovered an amazing image of what appears to be a magnetically powered spaceship:

As he was reading the book, the idea of “Crystal Punk” came to him – similar to Steam Punk, but mineral related. He imagined crystals growing out of technology and being utilised by people. He referenced Tiberium in the Command and Conquer series of games. I referenced the idea of the Cargo cult after an apocalypse or collapse – people not understanding how computers worked, but remembering the protocols around them.

I challenged Jayson on what his narrative was – is the idea that post collapse people are trying to cargo cult computers to make them work? Or do crystals become a commodity? Jayson stated that JG Ballard’s “The Crystal World” was his starting reference. The idea of post apocalyptic societies worshiping crystals.

I challenged Jayson on how the installation would love? Is he a Crystal Punk? Is he a future shaman? Is he going to make a performance? Is he going to look through crystals to see the future? Grow a crystal garden?

Jayson stated that he’s worried the project isn’t computational enough. That crystal lore is the thing that fascinates him. I suggested that the over saturation of computing resources could give rise to spontaneous crystal growth as an unexpected side effect. Or perhaps that a future AI grows crystals in order to create a physical form for itself – Conway’s Game of Life gone amok – cellular automata as a method for growing into the real world.

We then went online to search for other Crystal Punk references:

I suggested that he reach out to his course leader, Theo Papatheodorou to discuss the relevancy of his project.

We discussed what he would exhibit in his final show, and came up with the following elements (in no particular order):

  • A game of life simulation
  • Crystal punk artefacts
  • The narrative idea of computing waste getting to a critical point and an AI emerging at a certain point in time when cloud computing got to a certain density
  • A cargo cult of humans trying to understand what is going on? crystal lore?
  • Perhaps a space like an obsessive room of the man that realises that it’s going to happen – maps of all the data centres in the world and game of life of them spreading
  • A timeline of events

I referenced a cut scene from ET, where Elliot draws the circuit diagram of the customised Speak and Spell machine on the walls of the school nurse’s office:

Jayson talked about using a crystal to gaze into the future and seeing crystals everywhere. He found an amazing film from the 50’s that showed crystals growing everywhere: “The Monolith Monsters“:

I referenced Mark Lombardi’s work on networks of people and that he could use a similar visual style to show the progression of his work. Jayson admitted that he’s been dreaming of crystals! I suggested making a crystal mask to express that feeling. I also referenced Francis Bacon’s studio being relocated to Ireland from London after his death.

After the session, Jayson sent me an image of the beginning of his work on the crystal mask:

I set the following homework:

  1. Continue to experiment with Experiment with LEDs in the crystal growth structures and computer case – wax protection? Clingfilm?
  2. Send dissertation proposal and bibliography
  3. Send over the narrative of the space
  4. Speak to Theo about your concerns about computation relevancy