Voice Scripting / Intersecting Circles of Interest / Understanding SelfDecember 6, 2014
Everyone has things that they are strongly interested in and/or enjoy doing. A realization this week is that occasionally we come across an activity that combines 2 or 3 of our interests, and that can result in very strong levels of engagement. A way to visualize this is the venn diagram -- intersecting circles.
I experienced this phenomenon this week as I was setting up some voice-activated scripts/actions on my computer. I have things setup so that when I press Ctrl-Q, my computer starts listening (via Google's web speech API), and I can then issue voice commands. If I say "email John", my work email program creates a compose window and fills in John's email address. If I say "email Meredith using Gmail", a web browser opens a Gmail compose window and fills in Meredith's email address. If I say "lookup John in the company directory", the company directory page opens in my web browser with results filtered for "John". I could go on and on with the plans that I have for this technology... what is interesting is that setting those voice scripts up was mind blowingly compelling for me. I could easily lose myself for many hours, days, perhaps even months, implementing that kind of thing.
As I was setting this up, I recalled my days working for my former employer. Something that made me unique was scripting. I had a directory with 100+ scripts, many of them built on top of each other, allowing me to do all sorts of things with incredible efficiency.
It's no mystery to me that I'm hugely motivated by efficiency
. Those are two highly overlapping areas of interest. But upon further thought, there are several areas of strong interest at play with these voice scripts. It's fascinating to unpack those things, because it uncovers significant aspects of who I am, the things I value, what I find natural, etc.
|1.||Automation: Another way to put this is that I find repetitive tasks extremely frustrating. Any time in the past that I've had to do data entry, for example, it's like a "nails on chalkboard" experience. Working on a computer is no stranger to repetitive tasks, or tasks that consist of a bunch of predictable steps that just begs to be simplified down into one step. Any time I'm able to automate something, it is extremely rewarding. See also: Efficiency, Productivity. Creating voice scripts is a powerful form of automation.|
|2.||Efficiency: I've heard that my grandpa Green highly valued efficiency, and if so, then the apple didn't fall very far from the tree. My mind seems to naturally gravitate to thinking about how to do things more efficiently. As mentioned, there is a strong connection here to Automation. Another connection is that I really dislike waste, and inefficiency is wasted time. Creating voice scripts is a great way to make doing your work more efficient.|
|3.||Productivity: Strongly related with 'efficiency' and 'automation', I find high levels of productivity extremely rewarding. The todo list with ten items on it, all checked of as completely, is thrilling. Anything that helps me get work done faster is very compelling, and voice scripting fits this bill.|
|4.||Natural Language Understanding: The area of technology that I'm most interested in (and have been for about 6 years) is the problem of natural language understanding, and this is a key component of creating voice scripts, because it requires understand/mapping natural language commands to the actions you want to achieve.|
|5.||Voice Commands: Back in 2008, I wrote an application called "Grace" that ran on an iMac in the corner of our kitchen. In a sense, it was very much similar to the voice scripts I'm building now, but was focused on home life rather than work life. Having a computer be able to understand the spoken word was somehow very compelling and exciting for me, and still is. It puts a smile on my face.|
|6.||User Interfaces: When I was a teenager I discovered that I had a significant interest in graphical user interfaces. I loved the challenge of crafting a UI that would allow a user to get something done. One day I was contemplating my intense interest in natural language understanding, and it suddenly dawned on me that there was actually a connection between my interest in that and graphical user interfaces. That is, they are both user interfaces. Voice scripting is a fascinating addition to keyboard and mouse. My general sense is definitely not to replace keyboard and mouse with voice. I think that's a huge mistake. The idea is to supplement keyboard and mouse with voice, and to do that where voice is more powerful. For example, if I want to email Meredith using Gmail, and if I use keyboard and mouse to do it, I need to: 1) Move my mouse down to the Windows task bar, 2) Click on Chrome, 3) Press Ctrl-T to open a new tab, 4) Click on Gmail, 5) Click on 'Compose', 6) Type in Meredith's email address. That is 5 or 6 steps to initiate the desired action. Using a voice script, I have to: 1) Press Ctrl-Q, 2) Say 'Email Meredith Using Gmail'. The 2-step solution is cognitively much simpler and takes about half of the time. Meanwhile, if I'm typing an email and I want to type something in a bold font, would it be easier to have to press Ctrl-Q and speak "bold font" than it is to simply press Ctrl-B? No, of course not. It's all about choosing the right tool for the right job.|
|7.||Minimalism: It's no mystery to myself or others that know me well that I seem to be a minimalist by nature. I like clean, tidy environments that aren't overrun by junk. (There was another famous minimalist -- his name was Steve Jobs!) I love well designed things that have had all of the rarely-used knobs hidden away in cupboards, and execute brilliantly on the core functionality. Voice scripts are actually a pretty strong embodiment of this vision. There are no knobs to be bombarded with. You just say what you want, and it's done. What a beautiful minimalist UI!|
|8.||AI: At some point, as perhaps many technology people can relate to, I became a bit captivated by the idea of "AI". Probably because we all realize how powerful / useful intelligent systems are. Being able to create software that leverages intelligence is extremely compelling to me, and voice scripting intersects this realm.|
|9.||Cutting Edge: As with many technology people, I get a real kick from seeing and using things that are new and interesting, which push the limits of what is possible. Voice scripting has that flare to it for me.|
|10.||Creativity: I enjoy my work much, much more if there is a strong element of creativity involved. Envisioning how voice scripting should work, what use cases to target, etc, is a nicely creative process.|
|11.||Self-directed Work: Probably like most people, I find implementing my own ideas more interesting and rewarding than being told "go do this".|
|12.||Ideas: I'm an idea person. I find it thrilling to come up with ideas that seem valuable. I've never actually seen someone demonstrate a really effective synergy between keyboard, mouse, and voice, and so in envisioning that possibility, I feel like I'm tapping into a powerful idea.|
So this voice scripting stuff combines many different areas of strong interest for me. I suppose it shouldn't be surprising that setting up voice scripts is such a rewarding endeavor for me. It's such a strong intrigue that I find it hard to stop thinking about it.
The end-goal vision is this: Being able to do one's work incredibly efficiently with powerful voice scripts, which are backed by robust natural language understanding.
Want to start working on a task? Just copy the task ID into your clipboard and say "work on this task" -- your time tracking program will start logging that task, the task will be displayed in Jira, a new directory will be created for the task with the appropriate name, a new Mathematica notebook will be created and placed into that directory and displayed on your screen, the type of task detected from the Jira issue and the appropriate notebook template used so that the various functions you need to perform the debugging (etc) and right there in front of you, etc.
Done working on a task? Just say "resolve the task" and the Jira issue will be resolved, with the appropriate time spent recorded from your time tracking tool. Back in your time tracking tools, your work log will be appended with a line item indicating that the task was completed.
Need to comment on the issue you're working on? "comment on task"
Need to create a new issue? "create issue", "create bug", etc.
Need to chat with someone? "chat with John"
Need to send an email to someone? "email John and Amy"
Need to request a tag move? "request a tag move" and have your recent commit be automatically found and an appropriate email with "To", "Subject", and "Body" filled out. Just review and press "Send".
Need to restart a software component that would normally take a bunch of manual steps? Just say "restart <blah>" and have it all done automatically.
Need to go to a directory? "goto <blah>"
Need to download a new software build? "update build"
My sense is that there are many strong candidates for core / frequent actions to be packaged up nicely, allowing work to be done very quickly.
Part of the thinking here is that the human mind wants to be freed up to do the high level thinking, and not be burdened with endless clicking, navigating, etc. If all of the unnecessary low-level stuff can be factored away, it leaves the mind better able to focus on the problem solving, and hopefully, that results in a higher level of mental engagement, more passion for the work being done, etc.
One final closing thought here is that I work from home, so yabbering away at my computer all day is no problem. This wouldn't really work for people clustered around desks in an office environment. So the utility of this idea more generally might not be as great as I might hope.Mind, Consciousness, And The Many-Dimensional CanvasJune 27, 2014
Our universe is composed of various dimensions. Mostly commonly, we think of the three dimensions of space, but time is also understood as one of the dimensions of our reality. We can think of these dimensions as a kind of “canvas”. In the same way that an artist’s canvas can hold paint, the canvas of our physical reality holds matter and energy in its four dimensions. They are the “paint” of our reality, giving shape to the picture you see when you open your eyes.
But many of us sense that there is more to our universe than matter and energy. As humans, we experience our reality via what is typically called consciousness. Color, which starts its journey in our physical world as a wavelength of electromagnetic energy, enters our eyes, is converted into electrical impulses, and somewhere along the way, allows us to experience color. How we understand the essence of that experience, or how it comes about, remains an incredible mystery. We can ask: What is the color red, as seen by humans? So far, no one knows. No one knows how to understand the essence of watching a sunset, or standing on the shore while huge waves crash around us.
But there seems to be a fascinating connection between the canvas of our physical reality, and the canvas of our consciousness. That is, we can think of consciousness, too, as a kind of canvas, upon which our experience is painted. However, consciousness would seem to be comprised of many dimensions. Sight is distinct from sound, which is distinct from touch, which is distinct from formal thought, which is distinct from emotion. One way to view consciousness, then, is that each “sense” is its own canvas, its own dimension, but that those canvases are united as a super-canvas which we call our consciousness.
This gives rise to the sense, which many people have, that consciousness is part of the fabric of the universe -- just like space and time are part of the fabric of our reality.
We’re missing a piece here. When I talked about spacetime as a canvas, I made a distinction between the canvas and what was on the canvas. In that case, matter and energy were what filled the canvas. I haven’t made the same distinction for consciousness -- we have used that word both to refer to the canvas, as well as what is on the canvas at any one time, but perhaps we should distinguish between the two. And here, language may start to fail us. One alternate candidate word for the canvas itself might be “mind”, while we might call what’s on that canvas to be “the experience of mind”.
One hugely significant difference between spacetime and “mind” (which I will use to refer to the canvas of consciousness) is that the canvas of spacetime is unified. The entire universe represents a single, continuous canvas. Conversely, the canvas of mind, so far as we know, is broken into many discrete canvases. My experience is different than your experience. I have a private theatre, which you know little about.
This gives rise to an important question: Is the canvas of mind truly broken up into discrete canvases, or is that an illusion? Perhaps each of us simply projects onto our area of a much larger canvas that unifies us all.
And that possibility of a unified canvas is somewhat breathtaking. One way to envision that is a wall of TV screens at a department store. Each screen is projecting its own story, but is part of a larger surface that contains all of those experiences.
When we think of a unified, many-dimensional canvas that is imbued with each of our consciousnesses, and when we use the word “mind” to refer to that unified canvas, we can’t help but wonder where the intersection between that canvas and God is.
What comes to mind for me is the movie Matrix, which supposed that our reality was actually inside of a computer. And I think that’s why, upon watching that movie, myself and many others resonated at a deep level. We intuitively have this sense that we are part of something greater than ourselves -- a structure of mind that carries and supports each of our beings, and without which we could not exist.
Such ideas can seem both encouraging and challenging to our notion of God. On one hand, it matches the sense that God is all knowing and all feeling, because the canvas of all space, time, and consciousness holds everything. (how’s that for a take on the sense that all of our good and bad deeds are recorded in a book)
On the other hand, if God is simply a unified canvas that holds each of our minds, we are left with a being that doesn’t have its own will apart from us, and that is in sharp contrast to the way most people view God.
It is helpful, though, to realize that God need not be limited by this unified canvas, but rather to think of this canvas as possibly being part of God’s being. Or, alternatively, is the canvas not part of God’s being at all, and simply carried by God. Those are old questions, and perhaps we have some hints to muse one way or another.
Let me leave you with the main thought of this essay, said one more time: We are amazing creatures that dance and flit across an expansive and many dimensional canvas. We occupy the canvases of space, of time, and of consciousness, all intimately linked together to form a super-canvas. We are composed of matter, energy, and things which we can yet hardly grasp. Thanks be to God.“What is Consciousness” and “Does God Exist”?March 20, 2014
As is not entirely uncommon for me, I got thinking about the mystery of consciousness again today. What happened to strike me in this instance was that, as you may yourself have realized, it is fascinating that consciousness is both extremely observable and extremely unobservable. On one hand, we’re all consciously aware, and our sense of self and our ability to experience what we see, hear, and touch is one of the best-known things in the universe to us. On the other hand, science hasn’t been successful at being able to observe consciousness. There is no microscope that can see it. There is no test that can indicate whether something is conscious or not. There are no compelling theories about what it is, or how it works. And so, amazingly, many scientists dismiss consciousness as uninteresting or irrelevant.
So we have this amazing, incredible phenomenon in the universe, as profound (or more profound) than the three dimensions of space, or the dimension of time, and it is completely undetectable by science. Do you see where I’m going with this? Many scientifically minded people dismiss the notion that God exists, because there is no way to directly observe God. And that’s where consciousness becomes an interesting beacon of truth: It declares to us that there amazing, profound things in the universe which science has no ability to observe. Woah. That kind of opens the door, now doesn’t it.
A secondarily interesting, but strongly related thought is that consciousness tells us that there are potentially other incredibly profound aspects to the universe/reality that science is unable to detect. Things which humans may never be able to observe, may never know exist.
I want to play with this notion that consciousness is unobservable for a moment. Let’s say that there was another planet in the universe that had robotic life on it. Unconscious robotic life. These robots were able to procreate by making replicas of themselves, they were able to roll around and do things, and they were many times more intelligent than human beings, but they were just as unconscious as a calculator. One day, those robots came to earth and chatted with human beings. The question is: Would they be able to detect consciousness from observing our behavior? Would they see a certain glint in our eye and conclude “Oh my, there is something really profound going on inside of these creatures”. Or would they merely conclude that, yup, there’s another group of behaving robots out there in the universe, just like them.
Let's re-ask the question: Is there any link between consciousness and behavior? Another way to phrase that question is: Could you design a robot that looked and acted exactly like a person, but which was just that, an unconscious robot? In university, my sense was that in theory, I couldn’t see any obvious proof that such a thing was impossible, and so I leaned towards saying that “no, consciousness is experienced, but there’s no proof it loops back and affects behavior”. Sometimes later I realized there is actually quite a simple proof to the contrary that was right before my nose. The very fact that I was sitting there pondering the question was proof: My amazing, mystical, mind blowing experience of consciousness was causing me to sit there and reflect on it, and my sitting there and reflecting on it was behavior. I never would have guessed that the proof would be so simple, and concise.
Some people would say the same thing about the existence of God: That while we typically don’t observe God directly, you can see the glint in the eye of the universe. It’s as if we were those robotic explorers, peering into the eyes of a creation. Do we see the that there’s something amazing going on inside? Or do we conclude, “Nope, nobody’s home, just a bunch of behaving dust”.older >>