Do you think its possible for machines to control human life one day?

We are already surrendering control of our lives to machines bit by bit, from the ECU in your car to autopilot software to the health support machines in the hospital.

Isaac Asimov wrote the 3 laws of robotics:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

But that's sci-fi. In real life, there are no such rules. You can develop AI, robotics and internet computing to anything you fancy, unlike cloning. Nothing stops you from developing a monster robot or software that takes down everything connected to the internet, and probably a few countries along with it.

So should developers be subject to strict ethical rules? Who's going to police it and stop rogue developers from unleashing bots that create havoc in other people's lives?

More importantly, do you think the tipping point will happen? As in the day when machines and software lock out humans and start doing their own thing, out of control, and even impose control over us for our own good - the Terminator scenario? (actually self-replicating viruses and worms are already going out of control, causing economic damage...)

IMO, a machine does not need to simulate the human brain or be smarter than it to control humans. It just needs to be smart enough to take over missile launching systems, trip the power stations and shut off water supply.

Its very easy to deny humans control over such facilities. Just embed some code that changes people's passwords. When they're locked out, the machine will be self-operating until it runs out of juice... which can be decades if it draws power from nuclear sources.

A few months ago the US issued an internal security alert after discovering how easily their infrastructure can be crippled because of bad computer code... whether done intentionally or not. The scenario is easy to imagine from a programmer's point of view.

Actually public utilties do use the internet... using VPN to send telemetry, billing information and so on, especially when their plants are distributed. They use a public network like the internet for cost reason (the cost to lay your own fiber station to station, branch to branch is crazy) & the belief is that VPN technology is secure enough to stay private. But security and encryption is a never ending battle.

Weapon systems... most mobile launch systems, ships & submarines use encrypted RF or microwave. Again, the moment you broadcast a signal in the open, you open a window to intrusion. The security strength is not much different than VPN... somtimes its only as good, or as bad, as the password the operator use. Give a password hammer software enuf time and they can break in.

Proprietary systems as a wall, yes this can work but we must remember, these systems are rarely islands. To shut down a proprietary bank system, you don't need to shut down their computer. You shut down the power station that supplies power to the computer. Unless the station is operated by the bank, it falls under public domain and vulnerable to the usual public risks. You don't even need AI to break in.

Power stations are the most vulnerable because one small failure can lead to a nationwide cascade failure, like what Malaysia suffered in 1996.

This hidden interconnectedness between public and private domains is probably what caused the US DOD to issue their warning.

Yes I totally agree with you on that one becoz there are still people who keep their PIN number together with the ATM card in ther wallet.

Ok lets say I agree that power stations, banks, and military applications today are all hacker proof. Let me get back to the the main point of my question which is, should the developers of AI, robotics and software be subject to strict ethical rules about what they can and shouldn't develop? to prevent a terminator-style scenario from ever happening.

If it sounds far fetched, think of the restrictions they're already putting on cloning. Maybe people are afraid it might lead to Frankenstein so some countries actually impose restrictions on that science. If they can do that, wouldn't they eventually do the same to AI and robotics development too? And most importantly, do you think such restrictions would be justifiable?

The doomsday scenario can happen without guns or missiles. A war can be triggered by sabotaging the economy and public infrastructure. The intruder doesn't need simultaneous access to every computer to do this either. It just needs access to one machine, one weak point that can trip other systems and force exception routines to cascade the attack. Human chaos will do the rest.

On whether machines will eventually reach self awareness, I'd be interested to see how they define "self aware", see whether a home alarm using motion sensors can be classified as self aware. The question still stands though... should developers be allowed to go all the way and build intelligent systems without any ethical controls?

Thanks for posting the bbc link. Interesting article that reads like the beginnings of Skynet.

Would it be accurate to assume that artificially-spawned self awareness is going to be the same as human self-awareness?

Humans are carbon based. Computers are silicon based. If left in the wild, can we assume that a silicon-based brain will develop consciousness the same way as a carbon-based brain, and adopt the same priorities in its existence?

In programming, I can define desire as a measurable gap that need to be filled. For example, if I'm a battery powered robot and if my power level drops to crtitical, I can be programmed to look for a power source to recharge my batteries. It is no different than the human desire for food. Both eating and recharging batteries serve the same function: to recharge energy.

Now if I am a self-learning, self aware machine, I still have these gaps that need to be filled, these "desires." If survival is my prime goal and if humans block all my sources of power, I may force through those blockades even if it means harming a human in the process. The big problem will come when energy sources dwindle and man & AI machines have to compete for the same resource.

This post has been edited by Beastboy: May 20 2010, 05:36 PM

Himself rather than himself? Ooo it makes humans sound almost divine, lol.

A bit off track here but I'll go with Agent Smith of the Matrix when he said humans are like viruses. We land on a spot, sapu everything until there's nothing left, then we move on to the next spot. We are like a disease. So if the runaway AI develops into the tree-hugging sort, we'll start looking like a disease fit to be wiped out.

But again, I go back to my original question that's still unanswered. While AI and robotics development is at its infancy, would you support the ethical limitations on the science the way they did for cloning?

Ok I get u, the difference between wants and needs. I think wants will appear when organisms are pushed to compete for limited resources, the way men flash their toys when trying to attract a mate. Question is, will self aware machines behave the same way when put under the same circumstances.

Um... who is the TS guy I keep seeing ppl refer to in their posts?

---

Added on May 21, 2010, 11:10 amAh.. the first answer, thanks.

Personally I'll vote no. Since we've proven time and again that we cannot be trusted with our own future, it wouldn't matter who's in charge, a benevolent AI or some 3rd world dictator. Yes the smart android may not be that benevolent but at this point, if I had to choose between a rational-violent machine and an irrational-violent human, I'll take my chances with the machine.


This post has been edited by Beastboy: May 21 2010, 12:21 PM

No way! Hahaha

---

Added on May 21, 2010, 6:28 pmEh? Since nobody had ever seen an alien except in Aliens movie, would you also tell NASA to stop spending billions on the SETI program?

This post has been edited by Beastboy: May 21 2010, 06:30 PM

Yes, I know, and thanks for stating your stand. If you had read the original post title carefully, could you kindly point to me where did it say "Shall we discuss how the Terminator will kill us all, just like the movie?" It said, "Will the Terminator-style doomsday ever happen?" Do you know the difference? In English, style means, "In the manner of", not to be confused with "This is the real thing."

Now I notice you are a moderator and assume you have placed this prohibition officially as a moderator. If yes, then please delete this thread in its entirety and modify your TOS to include "You are not allowed to make reference to any movies that the moderators feel is not "real" in any threads in Lowyat.net." If you do decide to let this thread continue, then please withdraw your prohibition. Thanks.

Thanks for understanding.

First we have to define what consciousness is because apparently the psychologists, neurologists, philosophers and computer scientists don't agree on the definition.

Consciousness in Latin means "1. having joint or common knowledge with another, privy to, cognizant of; 2. conscious to oneself; esp., conscious of guilt". Let's try to apply it to a machine.

Cognizance: By being equipped with sound and motion sensors, I could say that a home alarm is aware of sound and movement. It fits this criteria.

Conscious to oneself: I can interrogate the home alarm system via internet and ask, what's your status? The system does a self check and reports back that everything's normal. No different than A asking B "Are you all right?" and B looking at his arms and legs and saying "I'm fine."

Conscious of guilt: If a human is taught to distinguish between right & wrong, he will develop a basis for guilt. There is a prerequisite conditioning. Similarly I can also program do's and don'ts into a computer and with that prerequisite it too will be able to distinguish between a do and a don't in its actions.

So from a linguist's definition, a computer can be conscious.

Now how do you define self-conscious?

This is a bit off topic but the sperm whale, elephant and bottle-nosed dolphin have a larger brain mass than an adult human but their minds are not at par with ours in terms of thinking, language, etc. Does the number of neurons really determine the characteristic of the mind or is it independent?

That sounds like a biocentric view of the universe. It is also the Buddhist view as I understand it.

Another view about brain vs mind.

No matter how you describe colour, a blind man cannot see the beauty of a painting. No matter how you describe sound, a deaf man cannot appreciate the beauty of music. "Reality" is shaped by one's biological sensory organs + the brain's signal processing capability, hence the notion that we perceive our universe biocentrically.

Our brains are like circuit boards and all boards generally work the same way. However we ascribe different values to colour and sound. Even twins raised in the same environment often have different favorite colours and favourite songs. In that sense, your world is not the same as my world.

How does 2 copies of the same circuit board (brain) ascribe different meanings to the same stimulant? Is it due to unseen micro variations on the boards, uneven sensitivities of sensory input, or something independent of the board altogether?



This post has been edited by Beastboy: May 27 2010, 10:16 AM

Sounds like why parents curfew their kids. Cannot go out after dinner ler...