AI and the rise of China: this is the New New World

Economics, Politics, Philosophy, History and Culture
Forum rules
Always add something of value to the discussion and have fun. Mind your language, no porn, no libel, no trolling and no personal attacks.

Please note, views expressed on the forum do not necessarily represent the views of Mises UK. the Mises UK Council, Mises UK Faculty or Mises UK members. Membership of this forum is open to anyone world wide who is interested in civil discussion.
User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Wed May 16, 2018 1:49 am

Below is a technical talk. Even though it is quite technical, I want to say that this is the most exciting development in Deep Learning/AI that I've seen in a long time. DeepMind introduced "neural Turing machines" (NTM) which are a neural-net based implementation of the classical Turing machine (which is the theoretical model of the general-purpose digital computer, such as the one you are using now, reading this page). The "Memory-Attention-Composition" network (MAC) presented in this video is much more tractable from the point-of-view of real implementation. An NTM can, in priciple, do anything that a MAC can, but the MAC is much simpler to describe, mathematically, and its "programmatic" behavior is much easier to reason about than NTMs. I rank this as the most exciting recent development because it introduces a component that has been glaringly missing from AI until now - some capacity to make indirect inferences based on a training-regimen that looks like standard AI training (supervised learning).



For good measure, here's a picture of the MAC cell:

Image

User avatar
Jon Irenicus
Posts: 162
Joined: Sun Mar 25, 2018 9:36 pm

Re: AI and the rise of China: this is the New New World

Post by Jon Irenicus » Wed May 16, 2018 6:20 pm

Meanwhile, idiocracy, here we come...
Former overlord of the original Mises forum.

User avatar
Tom Rogers
Posts: 149
Joined: Thu Mar 29, 2018 5:55 pm
Location: The Atheistic Pagan Avatars' Own County
Contact:

Re: AI and the rise of China: this is the New New World

Post by Tom Rogers » Fri May 18, 2018 5:18 pm

Fact: Machines cannot operate independently of human beings.

That being the case, there are three broad possibilities:

(i). Cyborgisation: human beings merge with intelligent machines, evolving the human phenotype.
(ii). Human devolution: humans become dependent on machines, devolving the human phenotype.
(iii). Human supremacy: humans remain masters over machines.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Sat May 19, 2018 2:46 am



These chips have the potential to massively speed up (and lower the cost of) the training and running of neural networks - potentially millions of times faster/cheaper than Google's formidable TPUs. Note that these chips do not rely on any speculative breakthroughs, unlike quantum computation which still has dozens of large theoretical and practical hurdles to leap.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Sun May 20, 2018 10:10 pm

Uh, Did Google Fake Its Big A.I. Demo?

They didn't fake it. But this headline/article is precisely the reason I have been waving the distress flag over the issue of AI since around the end of 2014. The lay person has no way to realistically assess what the AI is or is not capable of. This creates all kinds of potential hazards - legal, commercial, political, social, etc. Changes are coming down the pike so rapidly that it almost defies description. Since 2014, I've read dozens - maybe more than a hundred - papers in the ML/AI field (and related areas in CS). By the time I get up-to-date on one area of ML/AI, there are a half dozen new papers that have dropped. It's staggering. The tech reporting in this field is lagging at least 6 months to a year behind the curve; and only the big headliner technologies like AlphaGo or Duplex ever reach the awareness of mainstream audiences. There's so much more going on right now.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Wed May 23, 2018 3:39 am

So much happening, it's almost impossible to keep up...







I posted recently about Memory-Attention-Compositional networks - in my view, they are the most exciting development in AI since AlphaGo beat Lee Sedol. But I rank this presentation as a close second-place:



Anyway, I saved the best for last:

Learning to Optimize Tensor Programs

The title might not sound like much but when you realize that tensor programs are what power artificial neural networks themselves, you should be able to see the self-recursion here - we are using artificial neural networks to automatically optimize neural networks. Other, somewhat similar developments in the field are more long range and are enabling neural networks to design other neural networks. But the parameter space is so large that even Deep Learning methods begin to break down. Automatic hyper-parameter optimization is another promising area. But optimizing tensor programs is like pouring rocket fuel on all the other methods because it will make Deep Learning itself that much more effective. Think of AutoML as "automatic program search" and tensor program optimization as "compiler optimization" - the first problem is provably unsolvable (in the limit), the latter problem is not only solvable but we have a large body of expert work in this area that can be improved upon by Deep Learning systems. So exciting...
Last edited by Clayton on Sat Jun 02, 2018 6:12 pm, edited 1 time in total.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Sat May 26, 2018 3:27 pm

Uber’s Self-Driving Car Didn’t Malfunction, It Was Just Bad

How did this thing even get on the road?!

Position Map Regression Networks (PRN)

Semi-technical article, but well worth reading. What's particularly interesting is the problem-solving methodology. Instead of trying to teach a neural network to recognize faces from any position on a flat surface, we train the neural net to recognize a 3D orientation of a face based on its 2D appearance (just provide bunches of training pairs spread across a wide spectrum of faces). The neural net can then orient and infer the likely 3D structure of a face based on partial, 2D information, just as we can. Like the zero-shot-learning paper that I linked in a previous post, this methodology is clearly inspired by human/animal learning processes - by Nature. And it is amazingly effective. So, the more general lesson is this: by copying Nature's methods, we can cheat off Nature in order to accelerate the process of developing our own artificially intelligent and artificial life systems. The point might seem obvious enough, but we don't necessarily follow the implications in actual practice - robotics is a good example of this where we have artificial systems (e.g. Boston Dynamics) that maybe take a few lessons from Nature but are almost completely man-made. Instead, the shortest path to the coming future is to build systems that are as close as possible to Nature's already-worked examples, and then innovate from there.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Wed May 30, 2018 2:41 am



The video starts suuuuuuuuuuper slow, I recommend jumping to 25:00 or so. But it's pure gold from there on. This is the stuff that the empty suits on TV aren't talking about but which is going to actually matter as we begin to enter the AI revolution. Not all of it is intuitive, but it's not so counter-intuitive that we can't reach any useful conclusions about what is likely to happen. The "we need Universal Basic Income because AI"-mantra is a perfect example of the level of discourse that characterizes much of the discussion about the future. We need to start thinking smarter... soon, we will have no other option.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Thu May 31, 2018 3:30 am

AI-winter is coming

Link broken at the time of posting (was working earlier...) I disagree with the post, but it's well thought out. Also led me to this (only slightly technical) paper which presents many important counter-points to the current Deep Learning craze. Deep Learning (DL) is not a panacea. It is not an alchemical recipe for unlimited AI (just add pixie-dust, I mean, silicon). On its own, DL will not give us general-purpose AI. The true significance of DL is that we can transform the human-computer interface (HCI) from this extremely low-bandwidth, clunky monstrosity into an efficient, high-bandwidth interface without the need to resort to exotic, intrusive technologies like neural lace. The human-to-human interface uses quite small bandwidth (measured in bits) to convey enormous amounts of semantic information. This is possible because the human brain is like a compression-decompression utility. We compress an idea into a small number of movements of facial muscles, small motor muscles (body language), and the vocal cords. In turn, this tiny number of small movements conveys an enormous amount of information to the brain of the individual we are communicating with. DL is allowing us to re-architect the HCI so that computers can learn to decode natural human signals -- handwriting, speech, facial expressions, and so on -- and translate them into complex sequences of machine instructions. In short, Deep Learning is enabling us to build vastly more efficient (for humans) compiler tools, APIs, and so on. In turn, these more efficient tools will accelerate further advances in artificial neural-net technology and other AI technologies. DL is a powerful catalyst but it is not the end-product we are seeking.

User avatar
Clayton
Posts: 121
Joined: Sat Mar 31, 2018 4:24 pm

Re: AI and the rise of China: this is the New New World

Post by Clayton » Fri Jun 01, 2018 2:06 am

For all the hype, AI still hasn’t impacted the US economy

During a bull run, Jeremiad voices are more valuable than usual. I'm convinced that AI technology has reached a tipping-point and is about to transform the technology sector and much of the traditional economy besides. But it is still true that it hasn't had much of an effect on the economy as of yet.

Here are some robotic applications in the construction industry that are either already live, or soon to come online:


Post Reply