MoogFest Day 2 – Workshops and Martin Gore
Today I got the chance to do a few workshops that I had signed up for. The first session was done by NC State and took everyone thru doing some WebVR development using AFrame and a site called Glitch. Glitch allows you to setup a WebVR environment quickly and easily. Fun class, reminded me of the early days of ActiveWorlds and second life. I have a quick sample put together in the class. Check it out: https://mr-webvr-moogworkshop.glitch.com
The second workshop was immediately after lunch and was about building a synth in your web browser. Well not really true as it was really about setting up a development environment using The Progressive environment and p5js plug ins. Unfortunately this sessions overlapped getting to see Martin Gore from Depeche Mode and I was not above to do the whole session. The beginning I was in was very slow and a bit dry, so I am hoping to spend some time going thru the sessions afterwards. All the material is located at the following site.
It was great to see Martin gore from Depeche Mode receive his Moog innovation award. As the writer of Depeche Mode music for decades, his music is some of the best in the last 40 years. He did a conversation with the head of mute records. Really enjoyed it, even if he was not much of a talker. The picture above is when the CEO of Moog presented the award.
Every so often a session is absolutely nothing like the description, the session called “Federal Funding for Artists, Technology, and Emerging Forms”, was one of those. While an interesting talk, by a engaging speaker and artist, there was no discussion on funding or getting grants. I ended up heading for an evening of music afterwards.
Two artists were very good. One Spencer Zahn used the stand up bass to fuse jazz and electronica: 
The next one, Debit – was pure electronics :
Really enjoyed the evening at the Pinhook!
A slow start – Moogfest 2019 Day 1
Today is day 1 of Moogfest 2019. Got here in time to pick up my pass at 10am (when it opened) and they were still setting up the place. To say setting up the place, is a bit of an understatement. The didn’t even have the tent fully constructed. The VIP lounge wasn’t built yet, and the were trucks everywhere.
To be honest, this is not really unsurprising, given that they changed ticket companies within the last month… and the workshops were scheduled very very late this year. The content this year seems to be even lighter than last year.. but the good news is both Martin Gore (Depeche Mode), and Thomas Dolby are here.
I spent this morning look at the pop-up store hosted by Guitar Center. The coolest thing I saw this year so far is the following synth:
The name of the company is Folktek, the are using a mix of electronics, gold, and wood to create beautiful instruments. I didn’t get to hear it played yet, but I did sample a few other synths by others. It is amazing to me that you can get some many sounds out of these old school synths.
Looking forward to a ton of workshops tomorrow.. and getting to see Martin Gore get his award from the team at MoogFest!!
Tesla Autonomy Day (Day 1)
So today Tesla held their first Autonomy for analysts. They described their own customer neural network chip for self driving. Began in 2016, Dec. 2017 first chip from manufacturing , July 2018 – chip full production, Dec. 2018 – started to retrofit employee cars, March 2019 – model S and X added to new cars, and April forward in the new Model 3’s. So three years from design to scale in all their major vehicles.
The goals of the hardware team are:
The final chip looks likes this:
Straight forward board, graphics, power, and compute. The chip has redundant chips, for safety and security. And the whole thing fits behind the glove compartment. Adding the idea of redundancy to say that the chance of the computer fully failing is less than someone losing conscieness while driving.
The chips us a consensus algorithm to see if they agree before actually sending the command to the car.
14 nm FinFET CMOS:
Also has a safety system on the chip so it will ONLY run software signed by Tesla. The chip can handle 2100 frames per second. Developed their own Neural Network Compiler for the chip – not a big surprise here.
Results:
- Runs at 72w (goal was stay under 100w)
- Chip costs is about 80% the prior
- Can handle 2,300 framers per second (it is 21x faster than last chip) (NVidia is 21 TOPS(1) and they are at 144 TOPS)
According to Elon – “all Tesla cars being produced now have all the hardware required for full self driving.”
1) TOPS – Trillion Operations Per Second
Software discussion:
Andre created the computer vision class at Stanford. Has been training neural networks at Google, Stanford, and other places.
So the basic problem that the network solves is visual recognition. By explaining on training and back propagation explains how a network actually works. In order to help the training sometimes they will have a human annotate an image for things like lane lines.
This is key for when their are unusual results, night time, wet roads, shadows, etc. Key message is more data makes the networks better. And while simulation can help, the real world is needed to address the “long tale” of reality.
Using bounding boxes, Andre took us thru a concrete example of how the training works. Great example of fixing problems in learning, a bike on the back of a car. Had to use training sets to show bikes on the back of cars, to help it realize they are one item from a driving perspective, not two. Finding things on the road is also critical, so sourcing on the road, the networking needs to understand in this.
The software identifies when people have to intervene to fix things, capture that data and send it to Tesla, which fixes it and ads it to the unit test cases. Tesla can also request the “fleet” to send them certain types of data, i.e. people coming in from the right or left lane to center. They then run the new model in “shadow mode” to test it in production. They check the results, and then can deploy the new fixes by flipping a bit between “shadow mode” and production mode. This is basically A-B testing. This is happens regardless if the car is using auto-pilot or not. People’s driving is actually annotating the data for Tesla.
Augmented Vision in the vehicles will show you what the car is actually “seeing”. Path prediction is used to handle clover leafs on highways.
Love the comment that most of us don’t shoot lasers out of our eyes to see around, this is why visual recognition is critical for level 4 and level 5 self driving cars. Tesla is going all in on visual.
Elon believes they can be feature complete this year for self driving, for people to trust it by 2nd quarter next year, and year end 2020 for regulatory ready. Of course that will take longer to get regulators to actually approve it.
Final representation is about driving at scale with no intervention by Stuart:
How do you package the HW and Computer vision together to drive this a production at scale. They actually run multiple simulations at the same time, and use them to challenge each other to get to a single set of real world truth.
A deeper discussion of how shadow mode works.
Then Controlled Deployment:
Get clips to then play them back with new simulations.. and tune and fix algorithms.
Then confirm vehicle behavior and allow the system to validate it is working correct, And based on this data collection they will then decide when code is ready to be turned on.
So based on their lifecycle they are able to increase the speed of moving fixes into production.
Elon ended up talking about how things were added in for redundancy since 2016, and how the cars are designed to be robo-taxis since Oct. 2016. Cars prior to this will not be upgraded, as it is more expensive to upgrade a car, then just make a new car.
Reviewing of the master plan, says that Tesla is on track. Expect to have first Robo Taxis out next year, with no one in them. The challenge is for people to realize that the exponential improvement curve is getting us there faster than not. Biggest challenge is getting regulatory approval. Will become a Uber or Air BnB model, and where not enough people share their vehicles Tesla will deploy vehicles themselves.
Goal of Robo Taxis will be to smooth out the demand curve of the fleet.
If you lease a Model 3 you won’t have the option to buy it (based on this new model).
Current battery pack is design GE for 300,000 miles.. new model will be 1M miles with minimal maintenance.
Compared cost of owning / driving a car verses a Robo taxi. And then looked at gross profit of a Robo taxi.
Looks like Tesla will take this seriously, and they actually have a clause in their contracts that people can’t use the car with other “networks” so you have to use there robo taxi network.
Very interesting presentation, technology (HW, SW, and Scale) plus business models.
Conversation is not enough
In the last post I talked about how conversations are a powerful way of interacting with a computer. A conversation must have context and memory in order to enable this power. Simple commands and individual queries are good, but not sufficient to enable the UX of the future.
Now let’s look at augmented reality (AR). Since 2014, I’ve had a device that allows for a graphical overlay in the real world, but it was not AR. It many ways it was no different than current audio assistants, it had a set of limited use cases that were ok, but it had no real context. Google Glass is an experiment that was put out in the public long before it had the capabilities to be worth while. Over the years, Google has taken it out of the consumer space, and released an updated pair for the enterprise.
While the ability to overlay information on the real world is helpful, what Glass missed was the context of that world. Both Google and Apple now have improved this functionally in their various AR developer APIs. Anchoring a 3D model to the real world allows your to address use cases like visualization of new furniture in your house. This is a start, but you also need to understand lighting (which has been introduced in ARKit 2.0) and Physics (available in most gaming platforms, e.g. Unity).
Understanding the context of gravity and lighting, allows for interactions in the world that are natural. Natural interaction requires simulations that understand all of this, and more. One thing I’ve not seen to date is the ability to move a virtual 3D object around a room, and see it pass behind other objects in the room. All examples I’ve seen to date have the object bounce forward and backwards as you drag it around.
The other item that is needed is devices that not only show you the world, but allow you to interact. Many device makers require that you have some kind of device you attach to your hand (think of the Oculus). Microsoft’s Hololens 2 seems to have solved this one, but I’ve not had a chance to play with it in person yet.
We are getting there, but putting the conversation in the AR world is the magic that will bring this all together.
The power of conversations
When we think of personal assistants today, we think of call centers and home automation – “Hey Alexa – turn on the lights”, “I see that your phone number is 202-555-1212 – please press or say ‘1’ if you are calling from the number on your account”. This is not a conversation, this is a set of discrete commands that you could replicate with a button or a web service.
More advanced call centers may have a level of AI which will ask “Please describe your issue so ‘I’ can route your call to the appropriate team”. In this case the system looks for patterns or keywords – kinda like a web based search engine to increase the likelihood that they can route you to the right team. We are inching closer to a time where these systems will get the routing right every time.
But what if you could have an actual dialog? The system needs to not only understand the domain that you in, “trouble shooting your home router” or “booking a vacation in the middle of a series of business trips”. The current domain specific machine learning can get us closer to conversations, but they have issues when the conversation drifts outside of the domain of knowledge that it is trained in. These domains are the context where the assistant is relevant.
Having a conversation without context is just a simple set of stateless commands. The Internet was built on this technology, but as businesses have extended the capabilities of the Internet, we’ve had to maintain state. State allows for context, and this is were conversations must head to realize the power of assistants.
State also allows for domain shifts. If I understand that we are talking about a piece of manufacturing equipment, and suddenly you ask me about the technical skills of an employee, I need to keep track of the domain of prior parts of the conversation and shift to the new domain of employee skills. Understanding this shift, potentially even running separate conversational threads in parallel to see the relevance of each thread to the current part of the conversation, may allow us to increase seamlessness of the conversational shifts.
Seamlessly crossing domains across a conversation will allow businesses to take advantage of voice, and help us realize higher value of this new interface.
UX of the Future
I’ve been talking about the idea of how our computer experiences will be changing for years over at my podcast – Game At Work dot Biz. Much of the time I’ve talked about this in fits ands starts, and as such I don’t think I’ve done a good enough job of explaining my thoughts in detail. My goal of this blog post is to start the thought process of explaining why I believe we are in the middle of a major change to how we will be using computers in business over the next few years.
I believe that for most computer interaction we will be using a combination of Voice and Augmented Reality. It’s that simple. The key things that are driving this are incredible advances in AI on the edge, massive improvements in AR development tools, and the mass acceptance of tools like Alexa, OK Google, and Siri.
I am not talking about the simple usage we have today – “Alexa – Play Coldplay”, “OK Google – Turn on my Lights”, and “Hey Siri – Remind me to do something when I get home”. I am thinking of Industrial use cases, where a worker is elbow deep in a piece of heavy equipment and needs to look up something in an equipment manual. Instead of looking at a printed manual, or even a PDF on a tablet, the worker will say – “How do I adjust this valve?” And a 3D overlay will display directly on the piece of equipment and “show” the worker how to perform the task. In meetings, a always on listening service will capture meeting minutes, action items, and schedule follow-up meetings. And while driving down the highway, a “head-up” display will provide meaningful and contextual information about traffic routing and weather, tied to your appointment schedule.
I am not “yet” suggesting that we will get rid of programming and heavy data entry tasks with voice input, but we will revise our interactions with computers, to understand context and continuity so that voice will be like talking to your own personal assistant, and our environment will be reactive to what you are interacting with. My goal over the next few posts will be to explain how this will happen. Stay tuned.
Industrial Internet Consortium – First time attendee(s)
This week, I was glad to see my old friend and former fellow 8Bar teammate Ian Hughes, for drinks while he was in town. I found out he was attending the Quarterly meeting of the Industrial Internet Consortium in Raleigh, NC. Ian is now an analysts at 451 Research and focuses on AR/VR and the Internet of Things. IBM’s Watson IoT unit, where I work as a Strategist, is also a member. I hadn’t realized that the meeting was in town, and found out from Ian that no IBMers were at the meeting. So Tuesday morning I scrambled to make sure I could attend, and that I could participate in appropriate sessions.
The IIC, is an international consortium that is focused on industrial uses for the Internet of Things. As you can image there are tons of working groups, Technicial groups, and breakouts ranging in all kinds of topics. The big news this year is that the IIC and the OpenFog Consortium were merging. As businesses look at benefiting from instrumenting their assets and processes, AI is happening at all levels of the architecture, Edge, Fog, Local Data Centers, Cloud and Enterprise Data Centers. This means that figuring out the right architecture for any specific use case requires understand requirements, processes, and technical standards. By bringing together these two communities I am sure we will have a better understanding of how businesses should implement specific use cases.
As a first time attendee, I spent much of my time listening and reading. I did get to meet interesting technical and business leaders from many companies. And look forward to seeing what my next steps may be, as a member of the IIC.
Security and IoT
This year there have been so many reports about IoT devices being hacked, being misused, and being subverted. I think we are at a watershed when it comes to Internet of Things Connected devices. There are two major issues, which if not resolved, will crater the industry and drive a large spike of distrust into the industry.
First, most device manufacturers are not in the business to support their devices for the long term. We’ve gotten to a point in society were we have been trained to replace our technology every few years. PC’s are designed to last three years at most, phones are replaced every two years, and many software products are moving to subscription models, as consumers expect a never ending supply of updates and features, which standard upgrade pricing does not support.
Building a product, and the team you need to support it long term, requires planning and long term revenue streams. If we look at these trends, the desire to design hardware devices with long term security in mind, requires a fundamental shift in how those devices are conceived and charged for. The shift to subscription programs for some hardware is a great example of this change. Apple is doing this with their iPhones and Microsoft is doing this with their Surface laptops. Having cell carriers do this is great for carrier lock in, but doesn’t necessarily translate to the full value for the manufacturer.
Second, we need to have more consequential legislation for data privacy and security. If the only individual that is truly harmed by bad security is the end user or company, then the incentives to get it right by manufacturers is pretty much minimal at best. Many of the basic manufacturers continue to just pump out basic IoT devices, with no long term goal to support them. They make their money on the building and selling new devices. They do not have a long term revenue plan for any specific device, their long term plan is make newer devices.
While this manufacturer centric view of devices makes tons of sense as a manufacturer, the privacy and security concerns are only there in building reputation for manufacturing. While this will have long term brand impacts, many manufactures actually build for multiple other companies, and so they pass that band impact on to the company that white labels their devices. This passing the buck, causes even worse security policies, such as default passwords and open ports. The other bad practice that comes about is for some manufactures, to actually take advantage of the bad security to capture data themselves and exploit this data for additional revenue.
Hopefully in 2019, the value of GDPR will be realized as companies start having to deal with a legislation that takes security and privacy a bit more seriously. I don’t think GDPR is perfect, but it is a step in the right direction.
Why the HELL can’t Apple Fix iTunes?
The number of times that the iTunes library file has corrupted on my iMac is insane! I have a very large iTunes library. It consists of all the CDs, LPs, Cassette Tapes, and even some Reel-to-reels that I own, and have converted to digital. The library is about 23,000 songs and well over a thousand albums. With a library of this size, I have to have it stored on an external NAS, because I also have over a 1,000 TVs episodes and movies too. My NAS is a Drobo5N with 18TB of storage on it.
Here’s the issues that come up periodically in iTunes, and everyone of them drives me totally nuts.
- First, iTunes has never really done a good job of supporting NAS drives (or really any external drive, as I had these same issues when I had an Drobo directly attached to the iMac).
- Second, every so often, iTunes loses track of where a song is. This is shown by a small (i) symbol next to the item. If I try to play an album with songs with this symbol it will skip those songs. If I go in and double click on the song title, iTunes magically remembers the location and the song will play.
- Third, duplicate songs. If you ever go in and tell iTunes to consolidate your library, it will kindly duplicate the songs into it’s own file hierarchy, leaving the original there to. This makes sense from a safety perspective (not deleting the originals, BUT it will then (at times) add the duplicates back in. I think this is a legacy issue when you tell iTunes to monitor a directory.
- Forth, album art. I have spent a TON of time making sure that all of my album art actually matches the Albums I have. So if I converted an LP to iTunes, I take pictures of the LP so that I can match it up. The number of times that I’ve loaded my library only to see the Gray background with the Musical notes in the picture instead of my nice Album art is beyond count. I have used Doug’s wonderful scripts to actually put the album art into the songs, but since iTunes loses track of some of the songs, it doesn’t always work.
- And finally, Meta Data. I like to sort my albums in a very specific way. To that end, I have changes sort artist, sort album, etc. but it doesn’t seem to stick there either.
I have been using the iTunes Match service (which I love!) so that I can have my music library on all my devices when I travel, so I would really appreciate it if Apple would spend SOME time and actually fix bugs in iTunes, instead of tweaking the UI.