Session 13: Final discussions before show

In my final tutorial with Julianne we made a new layout for her p5.js visualisation as well as re-organising the code so that different messages could be displayed in different colours. She agreed to redo the voice synthesis code after she’d completed her dissertation. She ordered the Apple Trackpad that we selected together at our last session.

In my final remote tutorial with Diane we discussed her other teaching as well as future curating work. We talked about how she could make a slow panning film of a bitter melon that she found:

As she says on her blog:

Images taken of bitter melon, which for me gives the visual effect of bodily internal landscapes, such as the gut. I will do a slow panning video of this weird fruit, with added agar, gel etc…

I will then do some post production, altering colours etc, and then hope to created a generated mesh work which would appear as a layer on top of the vide. This mesh would connect brightest pixels, or certain pixel values that give the best effect. I found an example of this in ofBook, However they are dealing with a static image.

We decided on three objects for her show piece:

  1. A slow panning film of the above fruit
  2. Agar bags of microbiome bacterial content
  3. A poster of her process and thoughts – possible in digital form

 

In my final remote tutorial with Jayson we discussed the editing of the video he has created to detail the Crystal Punk mythology he has written, intercutting news content with various crystal growth macro photography.

Session 12: Posters, Ikea and a Trackpad

Jules, Diane and Jayson were all in attendance.

On Wednesday 19th July people started picking spaces for the group show.

We started with Diane’s homework from her previous session:

  1. Think about the narrative of the show – what does she need to make?
  2. Send the poster for the show she is curating
  3. What objects or scenarios are you going to explore in the show? Inpill/outpoo? Could people encode information in DNA as a secure backup? What happens when people die with this information inside them? What happens when they kiss or have sex? Could these be future secure methods of data exchange?
  4. Continue working on dissertation
  5. Create diagrams for explanations using p5.js.

She’s been working to a final draft of her dissertation this coming Sunday.

I N N E R  O T H E R is the title of the show that’s she’s been working on  as a curator. The image is a macro shot of an Indian squash:

The above image for the show is the style that she wants to follow for her final show at Goldsmiths. She’s interested in making it using p5.js – and using screens in a portrait format. I commented that this changes them into portals. She’s thinking about what colours she’s going to use and getting to an Alien look. I said it was fine to use animated gifs in her p5.js code – being pragmatic with her programming is fine. We talked about how the special effects team on Stargate ended up using a side shot of a stirred bucked of water:

We then discussed the narrative of her show – what new things does she have to make?

Think about the narrative of the show – what does she need to make? Diane referenced the capacitive sensing technology she used in her previous Cross Talk project. She discussed using capacitance to sense a touch and using that touch to create a generative visual – distorting the screen – even using a VGA cable to display the image generated when you play a sound through it. – Making a reference to the idea of collaboration between human and bacteria being perceived as similarly “wrong”.

I suggested using a triptych of three 16:9 forms – lots of art history around the triptych – the human, the bacterial, the blend or thesis, antithesis and synthesis.  Is the synthesis the visualisation of sound? The simulation aspects of the research she’s doing are referred to as “in silico” instead of “in vitro”. She’s considering casting silicon for the final show.

We then moved onto a discussion of what scenarios she wants to explore in the final show. She hasn’t factored in the previously discussed DNA encoding into her dissertation. She referenced the recent story about Madonna blocking the sale of her hairbrush because of concerns about her DNA being extracted. I suggested a show structure of:

  1. Autobiographical
  2. Dissertation
  3. Example objects or scenarios

She’s been looking at plinths to display things on, I said think about three minutes of time at each station, a three minute loop at each.

I said less kebab, more autopsy!

We discussed the display of three objects:

  1. Her science poster
  2. A live bacterial object
  3. Still macro images from the Indian squash she discovered

I referenced the IDEO business card project from 2001 – a deep dive into thinking about person identity in the future that resulted in a series of speculative designs in the form of business cards. I also suggested that she think about narrating some or all of the show.

I set the following homework:

  1. Full narrative for show
  2. First pass on content for each of the three
  3. Kit list of what things are going to be
  4. Think about the handout
  5. Think about sound/narration

Next was Jayson’s homework from the previous session:

  1. Get one Ikea object and attempt assembly
  2. Keep going on final kit list – especially around screens and printed output
  3. Experiment with splash screens and proxy injection on crystal wifi
  4. Continue working on timeline and map content
  5. Consider audio – what does crystallisation sound like?
  6. Think about selfies – how do you put visitors in a crystal world? How do they become Crystal Punks?

He got all the Ikea objects! Two Vittsjö’s and two Vittsjö desk parks too. He’s assembled them without the letter draw in each case. The tables are pretty long and narrow – he’s been experimenting with vinyl cutting to make text too. Ikea have stopped making them so he’s had to shop around.

In terms of the final kit list, he’s decided to use his own screens to fit into the book cases – it’s easier than using college stock, which is limited as it has to be shared between all students. The list is currently as follows:

  • Two bookcases
  • Two tables
  • Crystal Memory stick
  • Crystal Cable
  • Crystal Bluetooth earpiece
  • Crystal Hard drive
  • Crystal CD
  • Crystal Router (with faked his own wifi unit instead as he doesn’t have password for the original hardware)

He’s given up on captive portal on the Crystal Wifi (proper name for splash screen) which are generally built into routers. Instead hee’s made a little Processing server to serve up pages, he thinks it should work. I suggested that he’ll have to make a DNS server to do it properly. I also suggested  a plan b of a browser set up that crystallises content, we found the following links together:

He’s continued working a crystalisation timeline and map – he demonstrated a crystal map in Processing. He’s currently working on an animation or slideshow using the narrative we discussed previously.

I suggested the idea of using a series of fake news broadcasts, referencing the above title sequence for Romeo and Juliet from 1996. I also referenced two recent shorts from Neil Blokamp’s Oats Studios:

He decided not to add sound to the experience or selfies – he thinks the space is going to be dense enough already. We finished by discussing possible things for the audience to take away – we decided that crystals were too poisonous, but business cards from the future might be interesting.

I set the following homework:

  1. Full narrative for show
  2. Full kit list
  3. Think about a takeaway/handout

We finished by going through Jules’ homework:

  1. Continue working on speech interaction.
  2. Firm up projector and tablet models.
  3. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  4. Finalise kit list.

We started by looking at the swipe interaction and re-factored her code. I suggested using the Magic Trackpad from Apple instead of messing around with a separate Android tablet and having to connect that to another device.

I set the following homework:

  1. Try Apple trackpad
  2. Complete kitlist
  3. Try splitting into four quadrants
  4. Try different colours
  5. Reorganise the content
  6. Have the text read it out when you swipe

Session 11: Crystal Wifi, John Dee and Vittsjö

Previous to the session in Goldsmiths I checked in with Jules via Skype as she’s currently out of the country. She’s currently pushing on her dissertation before moving onto Hammer.js and OSC integration. I emphasised that it’s critical to get all technical connectivity issues out of the way as soon as possible. We found the following links which she’s agreed to take a look at before our next session in two weeks.

https://github.com/colinbdclark/osc.js/
https://www.jroehm.com/2015/10/a-simple-guide-to-use-osc-in-the-browser/
https://github.com/genekogan/p5js-osc

Unfortunately, Diane was unable to attend, so Jayson and I got down to discussing his homework from the previous session:

  1. Re-approach Theo about computational relevancy
  2. Continue with crystal growing around the computer case
  3. Get kit list worked out for show and consider the layout
  4. Ask the technician team about the availability of vitrines and plinths to display objects within
  5. Do some investigations about how light is cast through his crystals
  6. Continue to work on text based Game of Life simulation and how it can be incorporated into his show
  7. Look at museum displays for ideas on how to label his project and individual pieces

Jayson emailed Theo about the computational relevancy of his work – he got confirmation that he’s taking a good approach overall – he’s planning on having at least one screen at the exhibition display the experimentation he’s been doing around Game of Life style simulations. He’s very interested in the idea of petrified or crystallised data.

Jayson is planning on crystallising a Wifi router for the show, and we had a discussion about how he could have a customised splash screen that would display when people connect to it. I referenced the Star Wars ASCII Telnet project, but we realised that it would be technically too challenging to expect people to Telnet in a gallery context. We went on to discuss the possibility of doing a JS injection on every web page browsed through the Crystal Wifi, a legal man in the middle attack. It looks like Jayson will have to set up a proxy server, running locally.

The attempt to grow a crystals around a large computer case was a failure – he’d need a far larger container to grow in – with gallons of boiling liquid safely contained. He did have an interesting failure with an attempted crystallisation of an old mobile phone:

Jayson visited the Science Museum archive to view John Dee’s crystal amulet – something that took months of arrangement. He’s going to return next week in order to view some accompanying documents. He feels he could do a whole project on John Dee alone!

Quoting from the British Museum website:

John Dee (1527-1609) was an English mathematician and astrologer interested in divination using crystals and mirrors. This purple crystal was used for curing disease and predicting the future by looking for symbols or the ‘ghosts’ of people in the stone. Dee claimed that this crystal was given to him by the angel Uriel in November 1582, and that Uriel had instructed Dee and his assistant Edward Kelley (1555-1597/8) on how to make the Philosopher’s Stone – one of the goals of alchemy. The crystal was entrusted to Dee’s son, Arthur (1597-1651) who passed it on to Nicholas Culpeper (1616-1654) as a reward for curing his liver complaints. Culpeper was a physician and alchemist who used the crystal to try and cure illness, until 1651, when he believed a demonic ghost burst out from it.

Jayson has viewed the available display equipment (vitrines and plinths) from the Goldsmiths stores and has decided to attempt to customise a series of Vittsjö stands to make his own custom display for the graduating show. He did consider aluminium extrusions and 3D printed connections, but decided that he liked the look of the Ikea designs, especially with their glass inserts.

Reference for the installation look Jayson is aiming for.

In terms of content, he’s planning on displaying the following things:

  1. Four crystal objects: ethernet cable, bluetooth headset, router and hard drive.
  2. A map of the world with timeline of crystallisation narrative – he’s currently working out if these should be printed or displayed on a screen for maximum edit-ability up to show time.
  3. The game of life crystallisation visualisation
  4. I suggested a crystal mirror or display of his crystal mask in some way – something selfie-able. I referenced the seaside mask cut out displays that are endemic to English resorts. He is going to research something realtime, possibly using machine learning. I said that something full length – even only analogue would be great.
A previous installation of Jayson’s work.

He’s been trying different coloured lighting through his crystals – it turns out that white light is the only one that works – a specific colour tends to produce poor results. We discussed the possibility of making a crystallised disco ball – something that could effect the whole space. I set the following homework:

  1. Get one Ikea object and attempt assembly
  2. Keep going on final kit list – especially around screens and printed output
  3. Experiment with splash screens and proxy injection on crystal wifi
  4. Continue working on timeline and map content
  5. Consider audio – what does crystallisation sound like?
  6. Think about selfies – how do you put visitors in a crystal world? How do they become Crystal Punks?

Javascript: The Good Parts, adding icons for the Reactickles and fixing bugs in KeyboardFountain

After reading Javascript: The Good Parts by Douglas Crockford I decided to make some minor changes to the codebase of the suite:

  1. Instead of using the postfix increment operator ++ use += 1
  2. Instead of using the == comparison operator which uses type coercion , use the more reliable ===. Similarly for !=, use !==.

After making those changes, I screen grabbed each Reactickle to make an icon. Midway through the process I realised that it was difficult to tell the difference between some of the mouse based interactions and the keyboard interactions, so decided to make animated video icons instead of a still.

I used Screeny to be able to grab at a fixed pixel dimension of 256×256, but drawn at 128×128 in the main menu.

After adding the video thumbnails I spotted a bug with the KeyboardFountain – objects were being added to the matter.js simulation, but never removed.

First I made sure that the array holding all the circles was empty before I started adding to it:

 while(this.circles.length > 0) { // https://stackoverflow.com/questions/1232040/how-do-i-empty-an-array-in-javascript
 this.circles.pop();
 }

Secondly, I made sure the matter.js World had been cleared before adding new elements to it:

 World.clear(world);

Finally, I noticed that the circles in KeyboardFountain were being drawn incorrectly, overlapping with each other. I realised two things:

  1. I needed to set the ellipseMode to RADIUS:
    ellipseMode(RADIUS);
  2. I needed to draw the circles in the simulation in a new way because of this mode change:
    ellipse(0, 0, r); //previously ellipse(0, 0, r *2)

    The updated suite can be seen here.

Fixing bugs in the beta version of the suite

After making the first version of the full suite, I duplicated the project and set to work on ironing out bugs.

I added the title of the Reactickle that is currently running to the back button which allows you to return to the menu screen.

 this.updateButtonText = function(textToAdd){
 this.title = textToAdd+"\nClick to return to Main Menu";
 }

I used the “\n” special character to make sure that the text instructions were on the next line.

While looking at the discussion in the above link, I discovered that the text() command in p5.js has optional width and height parameters, to make sure that the text stays within a defined area. I implemented this throughout the code.

 this.draw = function(){
 textSize(characterSize);
 noStroke();
 fill(this.backgroundColour);
 rect(this.position.x, this.position.y, this.dimensions.x, this.dimensions.y);
 fill(this.textColour);
 text(this.title, this.position.x+5, this.position.y+20, this.dimensions.x-5, this.dimensions.y-20); //https://p5js.org/reference/#/p5/text
 }

Taking each Reactickle one by one, I started with KeyboardScalingCircleGrid.

On line 14 of KeyboardScalingCircleGrid.js, I had a console output, detailing the size of the array that held each of the ScalingCircles, one for each keyboard button I was taking input from. Repeatedly starting the Reactickle and going back to the main menu resulting in the following output:

KeyboardScalingCircleGrid.js:14 The size of the circles array is 216
KeyboardScalingCircleGrid.js:14 The size of the circles array is 252
KeyboardScalingCircleGrid.js:14 The size of the circles array is 288

It was clear that I needed to treat the array in a different way, by emptying it at the beginning of the setup method of KeyboardScalingCircleGrid.js. Looking in the Javascript basics part of the p5.js wiki pointed me at the MDN array reference page, which didn’t have a clear method. I found a StackOverflow post on how to empty an array in Javascript, and ended up using the following code:

while(A.length > 0) {
 A.pop();
}

I added this to every other Reactickle that had a similar array bug.

While testing every Reactickle to confirm that I’d expunged the array bug, I discovered that that both KeyboardSquares.js and KeyboardFountain.js used a centred rectMode() rather than a cornered one. I added a rectMode() change before each of their draw calls, as well as reseting to corner mode in the main sketch.js to ensure that the GUI was drawn correctly.

Next will be creating some icons for the Reactickles, rather than the current placeholder text.

 

Session 10: More Crystal Growing, Digestion as Computation and Speech Synthesis

We began the tutorial with Jayson’s homework from the last session:

  1. Continue to experiment with Experiment with LEDs in the crystal growth structures and computer case – wax protection? Clingfilm?
  2. Send dissertation proposal and bibliography
  3. Send over the narrative of the space
  4. Speak to Theo about your concerns about computation relevancy

Since the last tutorial, Jayson sent the following images of his experiments:

He’s been experimenting with different chemicals: copper sulphate, magnesium sulphate and aluminium sulphate. Magnesium sulphate aka epsom salt tends to grow in a fairly “clumpy” fashion, see his mask experiment from the previous tutorial. Copper sulphate is is preferred choice. He feels like he’s got to a good level of knowledge now of what works and doesn’t. Both vaseline and cling film didn’t work as a method of keeping the crystals away from certain area of his sculptural objects, as the crystal solution needs to be hot initially.

Dianne suggested trying to use latex after Jayson’s further experiments with taping failed. He has a variety of scalpels that he’s been working with, but it’s a very delicate operation. Another problem he has encountered is that most of his objects float – he’s been having to use a combination of plastic forks and tape to hold things in the baths of solution that he has been preparing.

Jayson has purchased a container of de-ionised water to prepare his solutions with – the first he ordered sprang a leak in the post unfortunately. Tap water has lots of calcium in it which ruins the growth of other crystals. His next growing attempt will be using the PC case that he showed in the previous session. He’s going to document the process thoroughly.

Jayson spoke to Theo in general about his work, but I suggested that he needs to be specific about his final project and his concerns about computational relevancy.

In between sessions I found the upcoming event featuring Yesenia Thibualt-Picazo’s work on Craft in the Anthropocene. Jayson responded by stating that he isn’t so concerned about materiality – he’s keen for the crystals to appear mysterious and alien. He wants the exhibition to make people feel like they witnessing an intelligence of some kind emerge – that super saturation of information gives birth to crystals, in the same way that super saturation of chemicals does.

Jayson sent me his draft essay on Crystal Media Archeology, which has the following opening paragraph:

This essay discusses the process of crystal growth in relation to the development of computers, technology and network culture and how the process of crystallisation is inherent across all kinds of complex systems. Using a hybrid approach of looking at these developments through the methodologies of Variantology (Zielinski, 2006) and Media Archaeology (Parikka 2010), I will discuss a different way of looking at the history and evolution of computers and computational systems. I propose that computers and computational systems are a type of crystallisation that has continued throughout history, this global crystallisation process started with the coalescing of atoms and particles to create complex forms of matter in deep time has continued throughout the history of the universe and is apparent now in the rapid growth and increased density of technology and data.

He also sent me his notes on his Crystal Punk mythology:

In the early part of the 21st century the endless digitisation and quantification of information led to a supersaturation of data. At this point in history there was around 44 zettabytes of data which was stored on enormous data centres and cloud computing servers that stretched all over the world. In the same way that the supersaturation of matter in a liquid causes the crystallisation of that matter the huge amount of density of information caused the crystallisation of data into physical forms. What began in a cloud server farm in china and spread throughout the infrastructure of the cloud and all its connected technology, in the same way a computer virus spreads throughout the internet this crystallisation spread throughout the internet and crystals emerged on cloud servers, hard drives, personal computers, laptops and even memory sticks throughout the world.

Both are shaping up well. He’s going to go on a trip to China soon and was keen to gather more photographic content there – his narrative is that Chinese server farms is where these new crystal first emerge.

We moved on to discuss what he is planning to exhibit in his show:

  1. A screen playing a video of an explanation of his crystal punk mythology – i.e. crystals have emerged, why?
  2. A series of artefacts with crystals growing from them – a phone, a computer and USB memory sticks. I asked how they would be presented? Jayson said that he was going to research vitrines to protect the delicate crystal forms.
  3. A display of crystalmancy tools – drawing on the Hopi practise of using crystals to see the future. He demonstrated a text based game of life simulation, I suggested that this might be a good way of showing how people start to incorporate these crystals into their every day life and customs.

I set the following homework:

  1. Re-approach Theo about computational relevancy
  2. Continue with crystal growing around the computer case
  3. Get kit list worked out for show and consider the layout
  4. Ask the technician team about the availability of vitrines and plinths to display objects within
  5. Do some investigations about how light is cast through his crystals
  6. Continue to work on text based Game of Life simulation and how it can be incorporated into his show
  7. Look at museum displays for ideas on how to label his project and individual pieces

We then moved on to Dianne’s work, beginning with her homework from the previous previous tutorial:

  1. Continue to work on dissertation with Helen Pritchard
  2. Research Markov Chain’s specifically for use within her exhibition
  3. Send me her proposal for her dissertation and the bibliography for it

Since the last but one tutorial, Dianne sent me her dissertation proposal, which has the following title:

HUMANn2: Making Kin with the Aliens Inside

or

Disembodied Data: Decoding our Inner Ecologies

with the following description:

Using the human micro-biome as a lens I aim to question our understanding, coding and decoding of the complexity of life be it computational or biological, questioning what are the definitions of life that would apply to the human or a digital entity. Investigating the ever-expanding organic digital interface, referencing DNA data storage, slime mold logic circuits, synthetic biology, etc.

Dianne related her research on Hidden Markov Model‘s (HMM’s) – that they are used within bioinformatics to attempt to extract gene and pathway information from huge amounts of short DNA/RNA chains, which is what is produced from an attempt to sequence a metagenome (genetic material recovered from environmental samples). Researchers are attempting to make in-silico DNA to make a model of how the human gut works based on this research. All of her research on this research is going into her dissertation. Her proposition for the show is to re-imagine contemporary science practise (specifically molecular biology) around the micro-biome.

We discussed that aliens from another world might see humans as mere carriers or husks for our gut biome. Dianne referenced the ENCODE (Encyclopaedia of DNA Elements) project, and pointed out that they are using computational systems to understand life, which means that our explanations for life will be computationally based. Just like computers, genes have loops within loops, subroutines all within a bigger operating system (the body). Life is computation and computation is life. This brought up a discussion about her previous research into slime computing and the like.

We then moved onto a discussion of what her show would comprise of:

  1. A biographical introduction – starting with her mothers work as a microbiologist.
  2. A biological introduction to the gut biome
  3. Explanation of computational techniques for understanding the biome
  4. Explanation around computational aspects of DNA
  5. Exploration of the future of computation – that it will be biological. Could this comprise of artefacts? Might people eat something to compute it in the future and then poo out the results? Dianne has an idea about using silicon tubes to make a sculpture, but also echo’ing the look and feel of scientific conference posters – she’s recently made one for an art show she’s curating. I referenced the bone gun from Existenz:

I set the following homework:

  1. Think about the narrative of the show – what does she need to make?
  2. Send the poster for the show she is curating
  3. What objects or scenarios are you going to explore in the show? Inpill/outpoo? Could people encode information in DNA as a secure backup? What happens when people die with this information inside them? What happens when they kiss or have sex? Could these be future secure methods of data exchange?
  4. Continue working on dissertation
  5. Create diagrams for explanations using p5.js.

We then went onto Jules’ homework:

  1. Get going in speech
  2. Send me dissertation proposal and bibliography, as well as project proposal
  3. Get going in matter.js (less important)
  4. Keep chasing the technical team for projector resolution and borrow Android tablet
  5. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  6. Make a kit list for the exhibition

Previous to the tutorial, Jules and I spent some time refactoring her code to allow her to track down why her speech synthesis work with p5.speech wasn’t functioning. The refactor was successful, and by the end of the tutorial proper we had her software reading out parts of the conversations between people that she’s collected up to now.

We then discussed her dissertation proposal, entitled:

The Age of Technological Love

She’s going to start writing the dissertation in earnest immediately. We decided that working with matter.js wasn’t necessary for this project and so re-prioritised work on getting swipe interaction working with the Android tablet she’s been given by the technician team. She’s going to get tablet after this tutorial and begin working with it. She’s also made progress on the projector selection, but hasn’t finalised it yet. We went through a possible kit list for her exhibition:

  • Projector
  • Plinth for tablet
  • Mount for tablet
  • Computer for projection
  • Speakers
  • Signal interconnect for all devices
  • Power for all devices

I reminded her not to forget any necessary adaptors – especially for projector/computer connection.

I set the following homework:

  1. Continue working on speech interaction.
  2. Firm up projector and tablet models.
  3. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  4. Finalise kit list.

Making the Fountain part of the physics simulation and creating a suite of Reactickles

After trying and failing to get a triangle to represent the fountain in KeyboardFountain, I reverted to using a rectangle, which functions nicely.

The new sketch can be seen here.

After talking to Wendy, we agreed that the next step was to get all the individual sketches working in a suite.

I created a new folder to contain the suite: Reactickles3, and started by trying to run KeyboardScalingCircleGrid within a larger p5.js sketch.

I created a new sketch.js in the Reactickles3 folder and copied KeyboardScalingCircleGrid.js into the same folder. After making the entire KeyboardScalingCircleGrid a class of its own, I was able to call its setup, draw and keyTyped methods from within my main sketch.js.

With that functioning, I started to test other Reactickles in this new structure, starting with KeyboardWorm, followed by MouseWorm, KeyboardSpringyCircles, MouseSpringyCircles, KeyboardSquares and finally KeyboardFountain.

Once those were working, I created an array of those Reactickles as well as a main menu screen and a return to main menu button. There are still some bugs in the system, but I’m going to save those for the next session.

Session 9: More Markov Chains, Speech Synthesis and Crystal Punk

Unfortunately, Diane was unable to make the session, but she did send updates on what she’s been working on recently. She’s been concentrating on writing her dissertation but also finding out more information about Markov Chains. She’s also been looking into Rebecca Febrink‘s Wekinator project. She’s hoping to use some of the data sets from the Human Microbiome project and using those to train Wekinator to turn out endless streams of Biome sequences – so that then never repeat. She’s going to explore possible outputs and bring them to the next tutorial.

Jayson, Jules and I had a discussion about how planning for the end of year show is going. Jayson went to a show discussion meeting with Atau Tanaka. This year they have both ground first floors of the exhibition space, but positioning within the spaces will be decided at the next meeting. Jules is keen to have a large dark space to show her projection. Jayson described how last year everyone drew onto a large floor plan diagram of the space, and then revisions were made. I suggested that it would be great if they could all be close to each other – that this would be better than all having disparate separate rooms.

As Jules was absent from the last tutorial, we started with her homework from the previous tutorial:

  1. Gather more content, dependent on Tumblr responses.
  2. Make a prototype with:
    1. p5.js
    2. Data files in json for conversations
    3. Physics based chains of conversations, similar to this Matter.js example.
    4. Voice synthesis with p5.js-speech.
    5. iPad swipes via osc using HammerJS and p5js-osc.
  3. Keep chasing the technical team for projector resolution
  4. Set up your own GitHub.

She’s still trying to get more content from other people, but is happy with working with her own content for now. We went through the setup of Git, Github source hosting and Github webpage hosting via Github pages. I pointed her in the direction of various tutorials on Github setup and Git in general. She’s successfully pushed to GitHub.

We took a look at saving her text content in external JSON files, but realised it was faster for her to save the data directly in the source code of her p5.js project for now.

Jules is still going through the setup of Matter.js and p5.js-speech. She’s been in Japan recently and noted that they have a very different form of online intimacy there – it feels faster there, whereas in the West people build slower. She found people using Tinder and Grindr depending on their preference, but the Pairs dating app was also popular.

In terms of her dissertation she’s been reading “Control and Freedom” by Wendy Chun again, which was her original inspiration for this project. She’s fascinated by the idea of aura from Walter Benjamin’s The Work of Art in the Age of Mechanical Reproduction. She stated that Tinder or other social dating networks diminish the aura of relationships – they aren’t your waking, walking life. The only way to get to aura is by sitting down with people in the real world and communicating with them.

Moving on to the coding of her project, I suggested prioritising speech synthesis first, then physics simulation and then finally interaction with a tablet computer to allow for the familiar swipe gesture to be used to navigate her curated dataset.

I suggested drawing out all the text to an image and then redrawing that every frame, rather than laboriously redrawing the entire ellipse of text on every frame. We went through the p5.js-speech library, which seemed straightforward, although she did follow up with me later stating that she’d been having some problems. We are going to have a separate coding only meeting before the next tutorial.

Finally we moved on to securing the Android tablet computer that she’s going to be leant for the exhibition. We found a suitable device to secure it. Finally we discussed how much explanation is going to be visible at the exhibition, something which still needs to be resolved. Is she going to be there to talk it through with visitors for the duration?

I set the following homework:

  1. Get going in speech
  2. Send me dissertation proposal and bibliography, as well as project proposal
  3. Get going in matter.js (less important)
  4. Keep chasing the technical team for projector resolution and borrow Android tablet
  5. Work on getting swipe interaction working via OSC using HammerJS and p5js-osc.
  6. Make a kit list for the exhibition

We then moved onto Jayson’s homework from the previous tutorial:

  1. Experiment with lasers, LEDs and other electronics in the crystal growth structures.
  2. Read the Curious Lore Of Precious Stones.
  3. Think more about the overall look of the exhibition.
  4. Send me his proposal for her dissertation and the bibliography for it

In between this tutorial and the previous one I had sent Jayson an article from the Washington Post on how human activity has caused a whole new layer of minerals to form around the world.

Jayson showed some of his experiments with trying to grow crystals around electronics components:

I suggested covering the contacts of the electronics with wax and then removing it using a hair dryer after the crystals have grown. He’s going to continue experimenting.

He ran out of copper sulphate, so he bought a 25 kg bag! It seems that it’s used in the sheep farming business. He’s also been thinking about covering an entire PC case in crystal, so has also bought one of those:

He’s also been experimenting with growing crystals on lasers, but has been having similar problems to those that occurred around LED’s. I suggested trying wrapping the components in clingfilm and then carefully removing afterwards if the previously suggested wax technique was ineffective.

We moved on to his thoughts on “The Curious Lore of Precious Stones“. Jayson has been steadily working his way through the 500 odd pages of fairly dense writings on crystal lore. He discovered an amazing image of what appears to be a magnetically powered spaceship:

As he was reading the book, the idea of “Crystal Punk” came to him – similar to Steam Punk, but mineral related. He imagined crystals growing out of technology and being utilised by people. He referenced Tiberium in the Command and Conquer series of games. I referenced the idea of the Cargo cult after an apocalypse or collapse – people not understanding how computers worked, but remembering the protocols around them.

I challenged Jayson on what his narrative was – is the idea that post collapse people are trying to cargo cult computers to make them work? Or do crystals become a commodity? Jayson stated that JG Ballard’s “The Crystal World” was his starting reference. The idea of post apocalyptic societies worshiping crystals.

I challenged Jayson on how the installation would love? Is he a Crystal Punk? Is he a future shaman? Is he going to make a performance? Is he going to look through crystals to see the future? Grow a crystal garden?

Jayson stated that he’s worried the project isn’t computational enough. That crystal lore is the thing that fascinates him. I suggested that the over saturation of computing resources could give rise to spontaneous crystal growth as an unexpected side effect. Or perhaps that a future AI grows crystals in order to create a physical form for itself – Conway’s Game of Life gone amok – cellular automata as a method for growing into the real world.

We then went online to search for other Crystal Punk references:

I suggested that he reach out to his course leader, Theo Papatheodorou to discuss the relevancy of his project.

We discussed what he would exhibit in his final show, and came up with the following elements (in no particular order):

  • A game of life simulation
  • Crystal punk artefacts
  • The narrative idea of computing waste getting to a critical point and an AI emerging at a certain point in time when cloud computing got to a certain density
  • A cargo cult of humans trying to understand what is going on? crystal lore?
  • Perhaps a space like an obsessive room of the man that realises that it’s going to happen – maps of all the data centres in the world and game of life of them spreading
  • A timeline of events

I referenced a cut scene from ET, where Elliot draws the circuit diagram of the customised Speak and Spell machine on the walls of the school nurse’s office:

Jayson talked about using a crystal to gaze into the future and seeing crystals everywhere. He found an amazing film from the 50’s that showed crystals growing everywhere: “The Monolith Monsters“:

I referenced Mark Lombardi’s work on networks of people and that he could use a similar visual style to show the progression of his work. Jayson admitted that he’s been dreaming of crystals! I suggested making a crystal mask to express that feeling. I also referenced Francis Bacon’s studio being relocated to Ireland from London after his death.

After the session, Jayson sent me an image of the beginning of his work on the crystal mask:

I set the following homework:

  1. Continue to experiment with Experiment with LEDs in the crystal growth structures and computer case – wax protection? Clingfilm?
  2. Send dissertation proposal and bibliography
  3. Send over the narrative of the space
  4. Speak to Theo about your concerns about computation relevancy

Session 8: Fleming, Markov Chains, Lore and 3D Printing

We started by going through Dianne’s homework:

  1. Keep thinking on question for dissertation
  2. What is the installation? Bioinformatics of the Biome?
  3. Speak to the Wellcome trust collection what do they have on the biome? What apparatus? Even just the glassware collection.
  4. Contact the Science Museum – about their work on the Biome.
  5. Meet with William Latham and Frederic Laymarie at Goldsmiths.
  6. Make contact with the Bioinformatics team at Imperial.

During the break, I’d sent Dianne the following image, taken from Rod Judkin‘s book, “The Art of Creative Thinking“:

Specifically, that Alexander Fleming‘s discovery of penicillin was the result of his desire to make paintings with bacteria.

I suggested that the the idea of scientific discovery enabled by artistic desire would be an interesting one for her dissertation. She said that she’d already been looking at the career of Joe Davis and I suggested she look at both Robert Rauchenberg‘s founding of Experiments in Art and Technology, as well as Robert Irwin‘s work with LACMA’s Art and Technology group. I also suggested she speak to the other Computational Art’s MFA tutor, Helen Pritchard, to see if she had more reference and thoughts around this area.

We then started discussing how the human biome is data related – how to “disembody” data, pun intended! Dianne referenced Donna Haraway’s work – specifically her idea of “making kin with the aliens inside”. We talked about how life has been understood through science, and how our understanding of life is seen through a scientific viewpoint, specifically referencing the Human Microbiome project. Dianne is interested in the computing around the project,
even the language used between bioscience and computing, the relation between life and computation and looking at it all from a materialist standpoint. The idea of a human being an assemblage of fractured stuff.

Dianne went on to reference Samantha Frost and her book Biocultural Creatures and the relation between matter and energy. Jayson suggested looking at the book Decoding Reality by Vlatko Vedral. I was particularly interested in the idea of Destruction ab Toto or Nothing from Something:

The information basis for creation ex nihilo. According to John von Neumann, starting trivially from an empty set of numbers an infinite sequence of numbers can bootstrap their way out. An empty set creates the number 1 by observing an empty set within itself which is enough of a basis for distinguishability. It creates the number 2 by observing an empty set within the second empty set and the number 1, and so on. Vedral sees this not as creation but as data compression, as every event of a reality breaks the symmetry of the pre-existing formlessness. Science is the process of describing a large amount of observed phenomena in a compressed programmatic way to predict future outcomes, and in this process of data compression science creates new information by eliminating all contrary possibilities to explain those phenomena.

Dianne had also been looking at DNA as a way of storing data, I said that it was important to situate her work within the body and with scientists.

Dianne stated that she wanted her installation to be about applying scientific methods to make art. For example, the Shotgun sequencing technique for DNA. I related this to cut up technique – but she stated that she’d been researching Markov Chains, which are widely used in many applications.

I suggested to focus on one method and concentrate on that, and where it could be applied. Dianne stated that she didn’t want to make a feitished set of scientific apparatus, more something relating to a data centric approach to bioinformatics and the digital/organic mix of that. I repeated that it was still important to relate this all to the body.

This meant that her previous research on glassware was no longer relevant, but she did reference the human genome books at the Wellcome Collection.

She hadn’t found anything more on the biome at the Science Museum, or had any further contact with William Latham.

Dianne attended a workshop with the artist Anna Dumitriu at Imperial and met a few scientists there. They were working on the medicine side of things, though she did make contact with a postdoc at Glasgow university who is interested in dialogue.

I set the following homework:

  1. Continue to work on dissertation with Helen Pritchard
  2. Research Markov Chain’s specifically for use within her exhibition
  3. Send me her proposal for her dissertation and the bibliography for it

We then moved on to Jayson’s homework from the previous session:

  1. Make a crystal radio.
  2. Research Germanium.
  3. Consider your new mythology – around undersea wrecks? Flooded dumps? Mining in the future?
  4. 3D print a support structure to grow crystals on.
  5. Meet a crystal healer / do an esoteric walking tour
  6. Think again about the analogue/digital mix of your installation. Situate your practise in your own life history and memories.

Unfortunately, he didn’t manage to get his crystal radio kit working, I suggested an alternative kit from Amazon.

It turned out that Germanium has many different uses. He couldn’t grow it in the way that he had for other chemicals, it is formed geologically and also used in dubious “Health Bracelets“. Quackery!

In the break he went on an esoteric walking tour of Covent Garden. Watkins was super new agey, but the Atlantis bookshop was much better. He visited the basement exhibition of tools, silver bullets and even a mermaid skeleton! The shopkeeper was dismissive of crystal healing, but suggested looking at astro archeology. He found two books of interest:

  1. A Little History of Astro-Archaeology
  2. The Curious Lore of Precious Stones

We went on to talk about his new mythology. Jayson has been spending alot of his time writing and is almost at a first draft of his dissertation. He’s using the approach of “Insect Media” by Jussi Parikka as a basis for his writing around the development of development of computers in relation to crystals.

His mythology is based around the idea that crystals are spreading everywhere (inspired by the “The Crystal World” by JG Ballard). I suggested the idea of future AI’s using people as structures for growing crystals or the idea of data creation as crystal growth. Jayson also referenced the classic nanomachine trope of Grey Goo.

Jayson found a page relating to gemstone myth and folklore. He’s trying to find someone with a research background in the area. He’s very interested in crystal gazing, Mayan crystalmancy, magic in Europe and also found this dissertation on “Cunning Folk and Wizards“.

Next Jayson showed his experiments with 3D printing support structures to grow crystals on:

This took about a day to grow. He’s going to experiment with adding lighting, LED’s and lasers to the structures. I suggested growing a 3D disco ball! Interestingly, they can be dissolved and regrown at will.

Next we talked about the analogue/digital mix of the final installation – he’s very excited about the use of the 3D printed lattice but needs to work on the mythology more. Are these made by future AI’s? Do they emerge from electronic dumps? I also suggested having a live tank so people could see the crystals growing.

I set the following homework:

  1. Experiment with lasers, LEDs and other electronics in the crystal growth structures.
  2. Read the Curious Lore Of Precious Stones.
  3. Think more about the overall look of the exhibition.
  4. Send me his proposal for her dissertation and the bibliography for it

Unfortunately, Jules was unable to make it, but she updated me on her Javascript work, especially around matter.js and voice synthesis.

 

KeyboardSquares and KeyboardFountain with Matter.js

The original KeyboardSquares can be seen above – on key presses the larger square broke up into smaller squares and flew apart, as well as changing colour. It was a relatively quick port to create, based on previously ported Reactickles. One sticking point was making sure that the larger square was always centred – even if the user changed the size of the window of the browser. Luckily p5.js has a windowResized() function to detect that. I also found some different options for emptying an array in JavaScript. The ported version can be found here.

The next Reactickle was Keyboard Fountain which can be seen from 01:43- 02:20 on the video above.

After creating KeyboardFountain in my local Git repository, I set about creating the “fountain” at the bottom of the screen. Conveniently, p5.js has an existing triangle drawing function, called triangle(). Quoting from the reference:

Description
A triangle is a plane created by connecting three points. The first two arguments specify the first point, the middle two arguments specify the second point, and the last two arguments specify the third point.

Syntax
triangle(x1,y1,x2,y2,x3,y3)

I created the following function to draw my Fountain:

function drawFountain(){
 var fountainWidth = 50; //50 pixels wide
 var fountainHeight = 50; //50 pixels high
 var translatedX = this.fountainPosition.x * windowWidth;
 var translatedY = this.fountainPosition.y * windowHeight;
 var redColour = color(0,100,100,100);
 fill(redColour);
 triangle(translatedX-(fountainWidth/2),translatedY,translatedX+(fountainWidth/2),translatedY,translatedX,translatedY-fountainHeight); //https://p5js.org/reference/#/p5/triangle
}

After I was happy with the drawing of my Fountain, I added interactivity by using the p5.js keyCode variable to allow the user to use the left and right arrows keys to move the fountain:

function keyPressed(){
 var moveAmount = 0.01;
 if (keyCode == LEFT_ARROW) { //https://p5js.org/reference/#/p5/keyCode
 if(this.fountainPosition.x >= moveAmount){
 this.fountainPosition.x -= moveAmount;
 }
 } else if (keyCode == RIGHT_ARROW) {
 if(this.fountainPosition.x <= (1-moveAmount)){
 this.fountainPosition.x += moveAmount;
 }
 }
 return false; // prevent default
}

However, this meant that the user had to repeatedly press the arrow keys to move, whereas I wanted the Fountain to keep moving as long as I held the key down. I found the keyIsDown function which allowed me to have the Fountain move in the way that I wanted:

function moveFountain(){
 var moveAmount = 0.005;

if (keyIsDown(LEFT_ARROW) && (this.fountainPosition.x >= moveAmount))
 this.fountainPosition.x -= moveAmount;

if (keyIsDown(RIGHT_ARROW) && (this.fountainPosition.x <= (1-moveAmount)))
 this.fountainPosition.x += moveAmount;
}

In order to get the particles flying out of the fountain in a realistic way, I new I would have to create some kind of 2D physics simulation – at the very least using basic projectile physics. In the spirit of keeping it simple, I looked online to see if there were any existing physics libraries that worked with p5.js. Thanks to the fantastic Coding Train by Daniel Shiffman, I discovered matter.js.

Daniel was also kind enough to provide source code for p5.js running with matter.js on his GitHub.

I initialised matter.js by duplicating most of Daniel’s code in a new function called setupPhysics, after declaring the necessary variables for matter.js to run:

//matter-js and p5.js integration based on https://github.com/shiffman/p5-matter by Daniel Shiffman
//also see https://www.youtube.com/watch?v=urR596FsU68 introduction to matter.js by Daniel Shiffman
var Engine = Matter.Engine;
//var Render = Matter.Render; // commented out as we are using p5.js to render everything to the screen
var World = Matter.World;
var Bodies = Matter.Bodies;
var Body = Matter.Body;
var Composite = Matter.Composite;
var Composites = Matter.Composites;

var engine;
var world;
var bodies;
var canvas;

function setupPhysics(){
 // create an engine
 engine = Engine.create();
 world = engine.world;

//make walls to constrain everything
 var params = {
 isStatic: true
 }
 var ground = Bodies.rectangle(width / 2, height, width, 1, params);
 var leftWall = Bodies.rectangle(0, height / 2, 1, height, params);
 var rightWall = Bodies.rectangle(width, height / 2, 1, height, params);
 var top = Bodies.rectangle(width / 2, 0, width, 1, params);
 World.add(world, ground);
 World.add(world, leftWall);
 World.add(world, rightWall);
 World.add(world, top);

// run the engine
 Engine.run(engine);
}

I changed keyTyped to trigger a new function, addCircle – as well as adding code to draw all the new circles that would be created:

 

function addCircle(){
 var params = {
 restitution: 0.7,
 friction: 0.2
 }
 var translatedX = this.fountainPosition.x * windowWidth;
 var translatedY = this.fountainPosition.y * windowHeight;
 var radius = 21;
 var newCircle = Bodies.circle(translatedX, translatedY-(fountainHeight), radius, params);
 circles.push(newCircle);
 World.add(world, newCircle);

//set a random velocity of the new circle
 //see http://brm.io/matter-js/docs/classes/Body.html
 //from http://codepen.io/lilgreenland/pen/jrMvaB?editors=0010#0
 Body.setVelocity(newCircle, {
 x: random(-5,5),
 y: -random(15,30)
 });
}

function drawCircles(){
 stroke(255);
 strokeWeight(1);
 fill(randomColour);

for (var i = 0; i < circles.length; i++) {
 var circle = circles[i];
 var pos = circle.position;
 var r = circle.circleRadius;
 var angle = circle.angle;
 push();
 translate(pos.x, pos.y);
 rotate(angle);
 ellipse(0, 0, r * 2);
 line(0, 0, r, 0);
 pop();
 }
}

function keyTyped(){
 addCircle();
 return false; //https://p5js.org/reference/#/p5/keyTyped preventing default behaviour
}

After adding drawCircles to my overall draw function, I had a working prototype, with source code available on the project GitHub as normal.

The next steps will be to add the fountain to the simulation, as well as code to make each circle random in size and colour.