Storyboards for the interaction

Thanks to the wonderful Tom Jennings, the project has some storyboards to help explain how to interact with project:

Deep receives a photo message from her friend Zen showing her posing in front of an amazing sculpture that appears to hang in the sky above.
Deep receives a photo message from her friend Zen showing her posing in front of an amazing sculpture that appears to hang in the sky above.
Intrigued, Deep accessed the website that Zen pointed her to.
Intrigued, Deep accessed the website that Zen pointed her to.
After the website loads, Deep finds she can pan her smart phone around like a window onto the world and even use her finger to drag and change the sculpture in real time.
After the website loads, Deep finds she can pan her smart phone around like a window onto the world and even use her finger to drag and change the sculpture in real time.
Deep finds an angle she likes and shares it with her father via a text message.
Deep finds an angle she likes and shares it with her father via a text message.
Deep’s father receives the message from Deep, an image and an instruction how to interact with the project via a feature phone.
Deep’s father receives the message from Deep, an image and an instruction how to interact with the project via a feature phone.
After checking on Google Maps, Deep’s father sends his longitude and latitude to the special number that Deep shared with him.
After checking on Google Maps, Deep’s father sends his longitude and latitude to the special number that Deep shared with him.
Deep’s father receives a text message in response, showing the sculpture in the sky above his local area. He likes it so much he shows it to Deep’s grandmother.
Deep’s father receives a text message in response, showing the sculpture in the sky above his local area. He likes it so much he shows it to Deep’s grandmother.
In a nearby square, the British Council projects the sculpture on a large wall. Locals can see the sculpture change in real time as users touch it on their smart phones.
In a nearby square, the British Council projects the sculpture on a large wall. Locals can see the sculpture change in real time as users touch it on their smart phones.

Final changes to the demonstration files for user testing

2016_11_10_macbookprograb

Dietrich Ayala added click and drag functionality to the project, via the Click Drag component, so after merging and pulling his changes to my local project folder, I installed it via:

npm install aframe-click-drag-component --save

I also edited my demonstration page for the project to make it much clearer which pages should be used for user testing in India next week.

Finally, I moved the virtual mountain a bit higher in the virtual sky to encourage users to look up.

Creating the release versions of the demo’s for user testing

User testing in India begins next week via the British Council so this week is my last chance to tweak the two demonstration files I’ll be submitting, one designed for iPhones and PCs without cameras and one designed for Android phones and Laptops with cameras.

On a Google Pixel phone the Mountain component isn’t particularly speedy, so I started by duplicating the component and trying to edit it to reduce it’s resolution. I also created a new pug file for the faster component and added it to my mainForRelease.js file so that Browserify would bundle it properly.

After a chat with the author of the component, Kevin Ngo, on the A-Frame Slack, he advised me to submit a pull request with the changes I required, and he would fold them into the main branch. He also supplied me with a tutorial on how to submit a pull request. I removed all my previous work and started by forking Kevin’s K-Frame repository by clicking the fork button on it’s GitHub. A new fork now existed on my GitHub. I then cloned it to my local machine:

git clone https://github.com/JGL/kframe.git

Then I moved into the folder itself, and created a new branch called mountain-variable-size:

cd kframe
git branch mountain-variable-size
git checkout mountain-variable-size

I then ran:

npm install

To install dependencies locally, then:

npm run dev

To ensure I had a local working copy. I ran:

git config --global core.editor "nano"

To make sure I was using the simple nano editor for any git editing that might happen. I then edited kframe/components/mountain/index.js to the changes that I wanted, which were all to do with reducing the number of width and height segment in it’s internal PlaneBuffer. I then added it to my branch, ignored the generated kframe.js and kframe.min.js files, committed my changes and pushed the changes to my fork on GitHub.

git add components/mountain/index.js 
git checkout dist/kframe.js
git checkout dist/kframe.min.js
git commit -m "Added world-depth and world-width to Mountain component"
git push --set-upstream origin mountain-variable-size

Kevin then merged the changes into his Master, and pushed the changes to npm, which meant I could just run:

npm update aframe-mountain-component

To get all the changes in my project. Creating a lower resolution mountain with the following pug code made everything run on the Pixel far faster:

a-mountain(id='mountain' color='rgb(128,0,128)' shadowColor='rgb(255,165,0)' world-depth='128' world-width='128' position='0 2000 0' material-side-modifier-mountain)

You can try the faster demo, and compare it to the slower version. I also added these changes to the release AR and the release static panorama demos.

Making my first A-Frame Component

Following Kevin Ngo’s advice, I decided to make a new component to allow me to modify either the landscape, ocean or mountain component’s material’s side property to allow them to be viewed from below. I started by creating a new file, material-side-modifier.js in my /src/js folder:

AFRAME.registerComponent('material-side-modifier', {
 //dependencies: ['yourlandscapecomponent'], // Or wait on an event.

init: function () {
 // Modify material side here.
 }
});

Then I modified my main.js to require my new .js file:

require('aframe');
require('aframe-terrain-model-component');
var extras = require('aframe-extras');
// Register everything.
extras.registerAll();
require('aframe-mountain-component');
require('kframe');
//requiring my first component! https://github.com/substack/browserify-handbook told me how to do this
require('./material-side-modifier.js');

Next I created a new .pug file, terrainModelComponentWithMyComponent.pug in my /src/pug folder, to use my new component:

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Terrain Model Component with Panoramic background
 meta(name='description', content='A-Frame Terrain Model Component with Panoramic background')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 //Terrain
 a-entity(id='landscape' position='0 100 0' terrain-model='DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;' material-side-modifier)
 //Sky
 a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient;')

I edited material-side-modifier.js to be as follows:

AFRAME.registerComponent('material-side-modifier', {
 //dependencies: ['yourlandscapecomponent'], // or wait on an event.
 // TODO: add waiting for an event to this instead of dependencies.
 // Thanks to @pookage (http://www.beardeddevelopment.co.uk/) on the A-Frame Slack for pointing me in the correct direction:https://developer.mozilla.org/en/docs/Web/API/GlobalEventHandlers/onload
 //Thanks to @dietrich () for suggesting changing from window.onload = changeMaterialSide; to document.addEventListener(‘DOMContentLoaded’, changeMaterialSide);

// Allow material-side-modifier component default to 2
 schema: {
 type: 'int', default: 2 //just a single-property schema, parse and stringify should be inferred.
 },
 // Update the side property of the material of the component to 2 aka THREE.DoubleSide, see: https://threejs.org/docs/#Reference/Materials/Material
 init: function () {
 console.log("A-Frame and the rest of the components have loaded");
 var object3D = this.el.object3D;
 console.log("Object3D is ", object3D);
 var side = this.data; //should be 2, the default value
 console.log("The value of data (side) is ",side); //why isn't this reporting properly?

//via http://stackoverflow.com/questions/18613295/how-to-set-a-texture-on-a-object3d-child-mesh
 console.log("object3D has: ", object3D.children, " children.");
 for(var i in object3D.children) {
 //for all the children of the landscapeObject3D, change the material side property to THREE.DoubleSide aka 2
 console.log("(Method 1: The current number of sides is", object3D.children[i].material.side);
 object3D.children[i].material.side = THREE.DoubleSide;
 console.log("Method 1: The updated number of sides is", object3D.children[i].material.side);
 }

// via: http://stackoverflow.com/questions/16027131/three-js-how-to-make-double-sided-object
 object3D.traverse( function( node ) {
 console.log("Traversing...");
 if( node.material ) {
 console.log("Method 2: The current number of sides is",node.material.side);
 node.material.side = THREE.DoubleSide;
 node.material.needsUpdate = true;
 console.log("Method 2: The updated number of sides is",node.material.side);
 }
 });
}
});

Accessing the new webpage I just created resulted in the following output:

build.js:70503 A-Frame Version: 0.3.2
build.js:70504 three Version: ^0.76.1
build.js:70505 WebVR Polyfill Version: 0.9.15
build.js:88899 A-Frame and the rest of the components have loaded
build.js:88901 Object3D is THREE.Group
build.js:88903 The value of data (side) is NaN
build.js:88906 object3D has: Array[1] children.
build.js:88916 Traversing...
build.js:38629 THREE.WebGLRenderer 76

Which suggested to me that neither of my loops to change the material properties are functioning as desired. Should I be using update instead of init? Or should I be waiting for a completely different event? Or do I need to specify custom dependencies? I’d really rather not do the latter as I want this component to be able to change the side property of any component that has a material.

After chatting on the A-Frame Slack, I found and filed a nasty bug for a Single Property Schema, and changed my component to use the Multi Property Schema syntax. I also found a silly mistake in my gulpfile.js, which I corrected, so that it watched for all Javascript file changes, rather than just main.js.

I simplified my component to the following:

AFRAME.registerComponent('material-side-modifier', {
 // This component can be used multiple times
 multiple: true,
 // Allow material-side-modifier component a single property schema, of type int, defaulting to 2, aka THREE.DoubleSide, see https://threejs.org/docs/#Reference/Materials/Material.side
 schema: {
 side: {
 type:'int',
 default: 2
 }
 },
 update: function () {
 console.log("Inside of update() of material-side-modifier");
 var side = this.data.side; //should be 2, the default value, all I want to be able to do is material.side = side; - change the side property of the material to
 var object3D = this.el.object3D;

 console.log("Starting traverse of object3D");
 object3D.traverse( function( child ){
 console.log("Traversing...")
 console.log("The current child object is: ", child);
 console.log("The type of the child is", typeof child);
 if ( child instanceof THREE.Group ) {
 console.log("Found a THREE.Group!")
 child.traverse(function(childOfChild) {
 console.log("Traversing the traversing...")
 console.log("The current child of child object is: ", childOfChild);
 console.log("The type of the child is", typeof childOfChild); //how is the mesh object always one away from me?
 if ( childOfChild instanceof THREE.Mesh ) {
 console.log("Found a THREE.Mesh!")
 }
 }
 );
 }
 }
 );
 console.log("Finished traverse of object3D");
 }
});

Then I found this issue on the Three.js Github where the author of three.js (MrDoob) states:

Geometries and Materials are not children.

Which means that the traverse() method of Object3D in three.js doesn’t iterate through the Materials and Geometries of Object3D’s. So I’m never going to get to my material by the above method. What method should I use?

After further discussions on the A-Frame Slack on the #Learning channel, I altered my new component to be as follows:

AFRAME.registerComponent('material-side-modifier', {
 // This component can be used multiple times
 multiple: true,
 // Allow material-side-modifier component a single property schema, of type int, defaulting to 2, aka THREE.DoubleSide, see https://threejs.org/docs/#Reference/Materials/Material.side
 schema: {
 side: {
 type:'int',
 default: 2
 }
 },
 tick: function () {
 var side = this.data.side; //should be 2, the default value, all I want to be able to do is material.side = side; - change the side property of the material to
 var object3D = this.el.object3D;
 if(this.el.getObject3D('terrain')){
 var terrain = this.el.getObject3D('terrain');
 terrain.material.side = THREE.DoubleSide;
 }
 }
});

As can be seen on this demo page, the component finally works, albeit in a way that is custom to the component it alters, the Terrain Model component.

Clicking on the Sculpture

In order to allow interaction with the sculpture, users are going to have to be able to click on it in some way.

Luckily, A-Frame provides the Cursor component to allow this. In order to familiarise myself with the component, I decided to follow the example provided in the docs, which allows the user to click to change the colour of entities that they click on.

To begin with, I created a new pug file (collidingWithTerrainModelComponent.pug) to create my demo html page, using the example from the docs and my previously developed component pug file:

doctype html
html
 head
 meta(charset='utf-8')
 title Colliding with the Terrain Model Component Demo
 meta(name='description', content='Colliding with the Terrain Model Component Demo')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 a-entity(cursor='fuse: true; fuseTimeout: 500'
 position='0 0 -1'
 geometry='primitive: ring'
 material='color: black; shader: flat')
 //Terrain
 a-entity(id='landscape' position='0 100 0' terrain-model='DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;' material-side-modifier cursor-listener)
 //Sky
 a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient;')

I created a new js file (cursor-listener.js), with the example code from the Cursor docs:

// Component to change to random colour on click.
AFRAME.registerComponent('cursor-listener', {
 init: function () {
 var COLOURS = ['red', 'green', 'blue'];
 this.el.addEventListener('click', function () {
 var randomIndex = Math.floor(Math.random() * COLOURS.length);
 this.setAttribute('material', 'color', COLOURS[randomIndex]);
 console.log('I was clicked!');
 });
 }
});

I also had to add the cursor-listener.js file to my main.js as a requirement:

require('cursor-listener.js');

I found that the demo was reporting a click, but it wasn’t changing the colour correctly, so I edited the cursor-listener.js file as follows, drawing upon the material-side-modifier.js component that I had developed previously. I renamed the file to cursor-listener-terrain.js as it was apparent that this component wasn’t going to work well with the way that the Terrain Model component was set up:

// Component to change to random colour on click, not functioning presently
AFRAME.registerComponent('cursor-listener-terrain', {

init: function () {
 this.el.addEventListener('click', function () {
 console.log('I was clicked!');
 if(this.el.getObject3D('terrain')){
 var COLOURS = ['red', 'green', 'blue'];
 var randomIndex = Math.floor(Math.random() * COLOURS.length);
 //this.setAttribute('material', 'color', COLOURS[randomIndex]);
 var terrain = this.el.getObject3D('terrain');
 terrain.material.color = COLOURS[randomIndex];
 console.log('I changed the colour!');
 }
 });
 }
});

In order to keep making progress, I decided to work with the Mountain Component that I had previously been working with. I renamed my previously made components to:

material-side-modifier-terrain-model.js
cursor-listener-terrain.js

And updated both the intial component demo and the cursor interaction demo’s to reflect the name changes.

I could then continue developing my colliding with Mountain Component demo and the component that would allow the initial colour changing interaction demo to occur. Because of more standard way that the Mountain component mesh and materials were set up, I found I could use the example code for the cursor listener exactly as in the documentation:

// Component to change to random color on click.
AFRAME.registerComponent('cursor-listener', {
  init: function () {
    var COLORS = ['red', 'green', 'blue'];
    this.el.addEventListener('click', function () {
      var randomIndex = Math.floor(Math.random() * COLORS.length);
      this.setAttribute('material', 'color', COLORS[randomIndex]);
      console.log('I was clicked!');
    });
  }
});

For the sake of completeness I created a demo of a similar interaction but using the Ocean component – I had to create a new modifier component to do that, but tried reuse my cursor-listener that I had used successfully with the Mountain component, unfortunately the Ocean component doesn’t seem to be identifying clicks. I filled the problem as an issue on the Ocean Component Github, and decided to keep working with the Mountain component.

A-Frame also has the Raycaster Component, so I decided to follow it’s tutorial to create a collider-check component, with the mountain component as it was the only component that had proved to be compatible with the cursor-listener component.

I created a new pug file (mountainComponentWithColliderCheck.pug) to to demonstrate the the newly created collider-check component. I also added the collider-check component to my main.js to that Browserify could do it’s magic. The resulting collider-check with Mountain component functioned as expected, as it output to the Javascript Console successfully when the cursor was over the Mountain component.

As the collider-check component was attaching a new listener to the Mountain Component itself:

AFRAME.registerComponent('collider-check', {
 dependencies: ['raycaster'],
 init: function () {
 this.el.addEventListener('raycaster-intersected', function () {
 console.log('Player hit something!');
 });
 }
});

I wanted to see if it would be possible to trigger the update() method of the Mountain Component. I created a new pug file (mountainComponentWithColliderCheckAndUpdate.pug) and component (collider-check-and-update.js):

AFRAME.registerComponent('collider-check-and-update', {
 dependencies: ['raycaster'],
 init: function () {
 this.el.addEventListener('raycaster-intersected', function () {
 console.log('Player hit something!');
 //console.log(this);
 //trying too call the update() method of the Mountain component to which this event listener is now attached
 update(); //this errors with Uncaught ReferenceError: update is not defined
 this.update(); //this doesn't error, but doesn't do anything
 this.el.update(); //this errors with Cannot read property 'update' of undefined
 });
 }
});

As can be seen in the comments above, none of my three methods for calling the update method of the Mountain component are currently functioning. I received a tip from DonMcCurdy on the A-Frame Slack:

@jgl: `el.components['component-name'].update()` should do the trick.

I also had to store a reference to the element before calling the callback, the complete code for collider-check-and-update.js is now:

AFRAME.registerComponent('collider-check-and-update', {
 dependencies: ['raycaster'],
 init: function () {
 var el = this.el; //tip from Don McCurdy, needed to store the element that this was called from
 this.el.addEventListener('raycaster-intersected', function () {
 el.components['mountain'].update(); //tip from Don McCurdy
 });
 }
});

You can try the demo for yourself here.

The interaction was rather clumsy, so I created a new .pug file (mountainComponentWithMouseAndTouchInteraction.pug) and added Touch and Mouse interaction to the entire page, so that clicking or touching anywhere would generate a new terrain – as well as adding a random colour generator for the color and shadowColor attributes of the Mountain component, and finally making the sunPosition attribute random too.

You can try that demo for yourself here.

The frankly amazing Dietrich Ayala created my first ever pull request for the project and added his getUserMedia() work to the project, to make the background panorama live on compatible (i.e. Firefox on Android) devices.

New Panoramas from India

r0010023

The fantastic Anne Siegel has just sent over some panoramas direct from the British Council location in Delhi.

In order to display them all in one convenient location I decided to use the A-Frame guide to building with components, which conveniently is a panorama browser.

Along the way, I had to install K-Frame:

npm install kframe --save

Unfortunately, I found some problems with the tutorial, so I posted a question to Stack Overflow.

Kevin Ngo, the author of the package helpfully got back to me, and bumped the version of his package. I edited my package.json to reflect his change:

    "kframe": "^0.3.2",

But this resulted in the following error when I tried to run:

npm update

To update K-Frame, the package in question:

ForceMacbookProRetina:APieceOfArtAsBigAsIndia joel$ npm update
npm ERR! Darwin 16.1.0
npm ERR! argv "/usr/local/Cellar/node/6.7.0/bin/node" "/usr/local/bin/npm" "update"
npm ERR! node v6.7.0
npm ERR! npm  v3.10.7
npm ERR! code ETARGET
npm ERR! notarget No compatible version found: kframe@'>=0.3.2 <0.4.0'
npm ERR! notarget Valid install targets:
npm ERR! notarget 0.3.0
npm ERR! notarget
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/joel/Documents/Projects/jgl/BritishCouncil/APieceOfArtAsBigAsIndia/npm-debug.log

I put the version number back to 0.3.0 for safety. Kevin got back in touch and suggested installing the Event Set component individually, so I ran:

npm install aframe-event-set-component --save

Which resulted in the following output:

a_piece_of_art_as_big_as_india@1.0.0 /Users/joel/Documents/Projects/jgl/BritishCouncil/APieceOfArtAsBigAsIndia
├── aframe-event-set-component@3.0.1 
└─┬ kframe@0.3.0
 └── aframe-event-set-component@0.1.2

Could this be a conflict? I added the following line to my main.js:

require('aframe-event-set-component');

When I tried to view the page on my local server, I got the following errors in my Javascript console:

Uncaught Error: The component `event-set` has been already registered. Check that you are not loading two versions of the same component or two different components of the same name.(…)

So after looking up how to remove a locally installed package via npm, I removed K-Frame by typing:

npm uninstall kframe --save

This removed all of K-Frame, as well as it’s dependencies. I could verify this by checking my package.json file. I now needed to add both the Template Component, the Layout Component and the Animation Component individually:

npm install aframe-template-component --save
npm install aframe-layout-component --save
npm install aframe-animation-component --save

And adding both of them as requirements to my main.js:

require('aframe-template-component');
require('aframe-layout-component');
require('aframe-animation-component');

Resulted in no errors on my Javascript console!

I continued through the rest of the tutorial, creating a pair of new components (set-image.js and update-raycaster.js), and including them in my main.js:

require('./update-raycaster.js');
require('./set-image.js');

I consulted the GitHub for the tutorial several times, but in the end I got the page working, allowing for the browsing of all of Anne’s images inside A-Frame!

 

Adding the sculpture to a panoramic image

Today I’ve been working on adding many different A-Frame components to the panoramic image I shot of my studio. First, I created a new .pug file combining the terrain-model component by Ben Pyrik with the studio panorama:

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Terrain Model Component with Panoramic background
 meta(name='description', content='A-Frame Terrain Model Component with Panoramic background')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 //Terrain
 a-entity(position='0 100 0' terrain-model='DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;')
 //Sky
 a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient; color: #0xeeeeee')

I added the demo my Github pages and took the following screenshot:

2016_11_01_panoramaandlandscapenotpositioned

I knew that I wanted the landscape to float in the sky above user’s heads, so I used the A-Frame Inspector to reposition the terrain model in the sky. Unfortunately, this had the side-effect of adding some glitches to the drawing of the terrain:

2016_11_01_panoramaandlandscapepositionedbutnotdrawingproperly

I suspect this is because A-Frame is expecting the terrain 3D model to be viewed from the other side. I decided to keep going with adding the other components I had found that could be good candidates for the interactive demo that I want to test with users in India.

The next component to try was Ocean, part of A-Frame Extras by Don McCurdy. I installed it via NPM as normal:

npm install aframe-extras --save

Then I added the component to main.js:

require('aframe');
require('aframe-terrain-model-component');
require('aframe-extras');

I then created a new .pug file, aFrameOceanComponentWithPanorama.pug:

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Ocean Component with Panoramic background
 meta(name='description', content='A-Frame Ocean Component with Panoramic background')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 //Ocean
 a-ocean(width='50' depth='50' density='40')
 //Sky
a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient; color: #0xeeeeee')

That didn’t display correctly, so I decided to build back up from the static demo’s that Don provided with the component, before re-adding them to the panoramic image. After downloading them, I managed to get them working on my local server by editing the following lines of his Water demo:

 <script src="https://aframe.io/releases/0.3.0/aframe.min.js"></script>
 <script src="https://cdn.rawgit.com/donmccurdy/aframe-extras/v2.6.0/dist/aframe-extras.js"></script>

But while isn’t my inclusion of the same component into main.js via Browserify working? I soon found the error, after re-reading Don’s readme.md. I had to add the following lines to register the component within main.js:

var extras = require('aframe-extras');
// Register everything.
extras.registerAll();

I added the demo to my GitHub, and took the following screengrab, with a similar glitch as before:

2016_11_01_panoramaandoceanpositionedbutnotdrawingproperly

Finally, I tried the Mountain component by Kevin Ngo. Again installing via NPM:

npm install aframe-mountain-component --save

First duplicated the demo to local system, which worked fine, so I created a new pug file:

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Mountain Component with Panoramic background
 meta(name='description', content='A-Frame Mountain Component with Panoramic background')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 //Mountain, seems to be on a large scale!
 a-mountain(position='0 2000 0')
 //Sky
 a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient; color: #0xeeeeee')

And added the following line to main.js:

require(‘aframe-mountain-component’);

To make sure that Browserify included the necessary JS files to make the webpage work. This resulted in the following demo and screenshot:

2016_11_01_panoramaandmountainpositionedbutnotdrawingproperly

After discussing my bug(s) on the fantastic A-Frame Slack it became clear that I need to edit the properties of the A-Frame Material which makes each entity an appearance. Specifically, I need to make the side property equal to double to render both sides of the mesh.

Bryvik helpfully made a new branch for his terrain-model component, that exposed the side property of the material, via the side property of the original three.js material that it uses.

As well as asking about my problem on the A-Frame Slack, I also browsed Stackoverflow for questions tagged with A-Frame. There I found a post about how to use JavaScript directly with A-Frame. The answer was written by the very helpful Kevin Ngo, but was written for the 0.2.0 release of A-Frame, not the latest stable 0.3.0. After consulting the docs for the 0.3.0 Entity I realised I also needed to work out how to write inline Javascript with Pug. Stackoverflow came to the rescue again – (Jade was the previous name of Pug) with a link to a Pug example of inline Javascript.

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Terrain Model Component with Panoramic background
 meta(name='description', content='A-Frame Terrain Model Component with Panoramic background')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true' stats)
 //Assets
 a-assets
 img(id='sky' src='firstPanorama/2016_10_18_FirstPanorama.JPG')
 //Camera
 a-entity(position='0 0 0' rotation='0 0 0')
 a-entity(camera look-controls wasd-controls)
 //Terrain
 a-entity(id='landscape' position='0 100 0' terrain-model='DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;')
 //Sky
 a-sky(src='#sky')
 //Light
 a-entity(light='type: ambient; color: #0xeeeeee')
 script.
 console.log("Hello world");
 var landscapeEl = document.querySelector('#landscape');
 console.log(landscapeEl);
console.log(landscapeEl.getAttribute('terrain-model'));

By looking in the Javascript console I could see that I was successfully printing “Hello World”, and setting the Landscape component to the variable landscapeEl. I could even make the Landscape visible or not by typing:

landscapeEl.setAttribute('visible', false);

or

landscapeEl.setAttribute('visible', true);

Into the console. I could also see that I was successfully displaying the public attributes of the Landscape component:

<a-entity id="landscape" position="0 100 0" terrain-model="DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;" rotation="" scale="" visible=""></a-entity>

As well as the attributes of terrain-model attribute itself:

DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;

– and side wasn’t one of them (I wasn’t using the branch of the Component that bryvik had made as I was trying to find a general method for changing the side property of any material in A-Frame. I realised that I needed to manipulate the three.js that underlies a-frame directly, which I could do by accessing Entity’s object3D property – which is a native three.js object. By adding the following

 var landscapeObject3D = landscapeEl.object3D;
 console.log(landscapeObject3D.parent);
 console.log(landscapeObject3D.children);

Via the console, I could then manipulate the THREE.MeshLambertMaterial side property (via THREE.mesh) directly, to get the mesh to draw correctly:

2016_11_03_partialsidewin

The next step is to do this programatically, but I’m running into a bug where I can type the command on the console to see the result, even trying different methods:

var landscapeEl = document.querySelector('#landscape');
 // Gaining access to the internal three.js object that the landscape component contains
 var landscapeObject3D = landscapeEl.object3D;
 console.log(landscapeObject3D.parent);
 console.log(landscapeObject3D.children);
 for(var i in landscapeObject3D.children) {
 //for all the children of the landscapeObject3D, change the material side property to THREE.DoubleSide aka 2
 landscapeObject3D.children[i].material.side = THREE.DoubleSide;
 }
 //this works when typed into the console, but not here programmatically, found here: http://stackoverflow.com/questions/18613295/how-to-set-a-texture-on-a-object3d-child-mesh
 landscapeObject3D.children[0].material.side = THREE.DoubleSide; 
 //this works when typed into console, but not here programmatically, found here: https://threejs.org/docs/#Reference/Materials/Material
 landscapeObject3D.traverse( function( node ) {
 if( node.material ) { 
 node.material.side = THREE.DoubleSide;
 }
 });
 //this also works when typed into the console, but not programatically, found here: http://stackoverflow.com/questions/16027131/three-js-how-to-make-double-sided-object

The A-Frame Slack came to the rescue, with pookage pointing out that I needed to call the code above only after the rest of the page had loaded, via the global JavaScript event handler: onLoad(). He also recommended the following the FunFunFunction YouTube Channel for general JavaScript learning.

 function changeMaterialSide(){
 //testing that I can print to the console
 console.log("A-Frame and the rest have loaded");
 //Gaining access to the landscape element via it's ID
 var landscapeEl = document.querySelector('#landscape');
 // Gaining access to the internal three.js object that the landscape component contains
 var landscapeObject3D = landscapeEl.object3D;
 //console.log(landscapeObject3D.parent);
 //console.log(landscapeObject3D.children);
 for(var i in landscapeObject3D.children) {
 //for all the children of the landscapeObject3D, change the material side property to THREE.DoubleSide aka 2
 landscapeObject3D.children[i].material.side = THREE.DoubleSide;
 }
 //this works when typed into the console, but not here programmatically, found here: http://stackoverflow.com/questions/18613295/how-to-set-a-texture-on-a-object3d-child-mesh
 landscapeObject3D.children[0].material.side = THREE.DoubleSide; 
 //this works when typed into console, but not here programmatically, found here: https://threejs.org/docs/#Reference/Materials/Material
 landscapeObject3D.traverse( function( node ) {
 if( node.material ) { 
 node.material.side = THREE.DoubleSide;
 }
 });
 //this also works when typed into the console, but not programatically, found here: http://stackoverflow.com/questions/16027131/three-js-how-to-make-double-sided-object
 }
 //Thanks to @pookage (http://www.beardeddevelopment.co.uk/) on the A-Frame Slack for pointing me in the correct direction:https://developer.mozilla.org/en/docs/Web/API/GlobalEventHandlers/onload
 window.onload = changeMaterialSide;

With this new knowledge, I added the material changing functionality to the other two component demo’s I had made before, making a total of new three pug files, which compiled to the following HTML/A-Frame pages:

  1. Terrain Model Component Demo
  2. Ocean Component Demo
  3. Mountain Component Demo

While 3. worked fine, after uploading, I tested 1. and 2. and found that the meshes would not render properly – even though 2. was rendering properly via my local server. I tried adding a call to Material.needsUpdate() after finding a Stackoverflow page alluding to it, but to no avail, even after changing the Material properties to just BackSide sides.

Finally, I posted a Stack Overflow question about it, where the ever helpful Kevin Ngo advised writing a component for the functionality, which I’m going to do next.

Adding Browserify to the project

When looking at both the HeightGrid Component by andreasplesch and the Terrain Model Component by bryik I noticed that both referenced a project called Browserify.

Browserify lets you require(‘modules’) in the browser by bundling up all of your dependencies.

In other words, you can use Browserify to include lots of different JavaScript components and package them up into one JavaScript file that can then be easily included in your projects.

To start with, I decided to try to re-make the A-Frame Terrain Model Component – Olympic Peninsula that I had previously duplicated from bryik.

First of all, I had to install Browserify via npm globally:

npm install browserify --global

Second, install both A Frame and A-Frame Terrain Model Component via npm locally:

npm install aframe --save
npm install aframe-terrain-model-component --save

Third, I created a new JavaScript file (main.js) in a newly created js folder inside my previously created src folder:

cd src
mkdir js
cd js
touch main.js

Fourth, I added the following content to my new main.js file:

require('aframe');
require('aframe-terrain-model-component');

Fifth, I created a new .pug file (aFrameTerrainModelComponentOlympicPeninsulaGeneratedViaLessAndPugAndBrowserify.pug) in my previously created pug folder inside my previously created src folder, with the following content:

doctype html
html
 head
 meta(charset='utf-8')
 title A-Frame Terrain Model Component - Olympic Peninsula made via Less and Pug and Browserify
 meta(name='description', content='A-Frame Terrain Model Component - Olympic Peninsula made via Less and Pug and Browserify')
 script(src='build.js')
 link(href="style.css", rel="stylesheet", media="all")
 body
 a-scene(antialias='true')
 //Is this assets line even necessary?
 a-assets
 //Camera
 a-entity(position='100 40 70' rotation='0 30 0')
 a-entity(camera look-controls wasd-controls)
 //Terrain
 a-entity(terrain-model='DEM: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/clipped-envi.bin); texture: url(aframe-terrain-model-component/docs/Olympic-Peninsula/data/olympic-texture.jpg); planeWidth: 287; planeHeight: 151; segmentsWidth: 287; segmentsHeight: 151; zPosition: 50;')
 //Sky
 a-sky(color='grey')
 //Light
 a-entity(light='type: ambient; color: #0xeeeeee')

Which was created by converting the original HTML source code, liberal use of the JavaScript console and the Pug API reference, particulary the attributes section.

Sixth, I could now check that my new .pug file was correctly formatted by running the following command in the root of my project:

gulp pug

Unfortunately, the newly generated html file wasn’t loading yet as build.js didn’t exist, so it was time to add a new task to my gulpfile.js, so that Browserify could create it:

// Modules
var gulp = require('gulp');
var pug = require('gulp-pug');
var less = require('gulp-less');
var browserSync = require('browser-sync').create();
var source = require('vinyl-source-stream');
var browserify = require('browserify');

// Tasks
gulp.task('default', ['pug', 'less', 'browserify']);

gulp.task('pug', function(){
 return gulp.src( './src/pug/**/*.pug')
 .pipe( pug( {pretty: true}))
 .pipe( gulp.dest('./docs/'));
});

gulp.task('less', function(){
 return gulp.src( './src/less/**/*.less')
 .pipe( less())
 .pipe( gulp.dest('./docs/'));
});

gulp.task('browserify', function() {
 return browserify('./src/js/main.js')
 .bundle()
 //Pass desired output filename to vinyl-source-stream
 .pipe(source('build.js'))
 // Start piping stream to tasks!
 .pipe(gulp.dest('./docs/'));
});

// Watching
gulp.task('watch', function(){
 browserSync.init({
 port: 4000, //where is browser sync
 proxy: 'http://localhost:3000/', //what are we proxying?
 ui: {port: 4001}, //where is the UI
 browser: [] //empty array of browsers
 });

 gulp.watch('./src/pug/**/*.pug', [ 'pug'])
 .on('change', browserSync.reload);

 gulp.watch('./src/less/**/*.less', [ 'less'])
 .on('change', browserSync.reload);

 gulp.watch('./src/js/main.js', [ 'browserify'])
 .on('change', browserSync.reload);
});

In order to make this new task work, I also had to install vinyl-source-stream via npm locally, as well as browserify locally:

npm install vinyl-source-stream --save
npm install browserify --save

I could see the files running locally by running commands that I first used in a previous post:

nodemon app.server.js
gulp && gulp watch

I’ve pushed all these changes to my GitHub for the project, and you can see the generated html file here, it’s identical to the file I duplicated manually before.

The following tutorials were useful as reference for this post:

Google Pixel, Making a Screen Recording of Dietrich Ayala’s AR Demo, Google Cardboard partial win

Today my Google Pixel arrived in the post, along with a pair of Google Cardboard. I’ve also got a Google Daydream on order, but it looks like it will arrive too late to be useful for user testing.

First impressions were great, it’s certainly a very capable smartphone. After successfully running the A-Frame AR demo by Dietrich Ayala on Firefox for Android, I wanted to be able to screen record it so that the A-Frame community could see the framework’s performance on Google’s flagship phone.

I downloaded Android File Transfer as well as Android Studio to enable me to connect directly to the phone on my Apple MacBook Pro. Helpfully, Google have provided a command line utility called screencap that allows for direct screen recording. Screencap runs within Android Debug Bridge (ADB) which needs to be enabled in the following way:

To access these settings, open the Developer options in the system Settings. On Android 4.2 and higher, the Developer options screen is hidden by default. To make it visible, go toSettings > About phone and tap Build number seven times. Return to the previous screen to find Developer options at the bottom.

After pressing Build number seven times(!) I was able to enable debugging mode on the new Pixel phone. This tutorial was handy to describe how to actually run ADB on my computer, I had to start a terminal in the following folder: /Users/<user>/Library/Android/sdk/ and then run the following command:

./adb devices

I actually had to run it twice, one to get permission to connect to the Pixel, and once again to actually get it to connect and report the serial number of the Pixel.

After playing around in the command line for some time, I was finding it very difficult to actually save a file to the phone’s local filesystem:

ForceMacbookProRetina:platform-tools joel$ pwd
/Users/joel/Library/Android/sdk/platform-tools
ForceMacbookProRetina:platform-tools joel$ ./adb devices
List of devices attached
FA69Y******* device
ForceMacbookProRetina:platform-tools joel$ ./adb shell screenrecord demo.mp4
Unable to open 'demo.mp4': Read-only file system

After doing some more searching I found a great app called AndroidTool by Morten Just that enabled me to record screen captures (and even convert them to animated Gifs) with just one click. The results can be seen below:

The final step was to try running the demo with Google Cardboard. I’m pleased to say that it worked great in terms of a 3D effect – but unfortunately, Cardboard physically covers the forward facing camera as well as the VR mode disabling the live video. I managed to get Firefox running fullscreen in non VR mode by using an addon, but it would be great to have VR mode not disable the camera.

2016_10_24_firstcardboard

Below is a screenshot of the demo working in Firefox for Android:

2016_11_01_aframearonfirefoxonandroid

But when you enable VR mode by pressing the icon in the bottom right hand corner of the screen, the live camera background is no longer displayed:

2016_11_01_aframearonfirefoxonandroidinvrmodecamerafail