Born in Melbourne, Australia to an English father (Len Williams, founder of The Monkey Sanctuary in Cornwall, England) and an Australian-Chinese mother (a daughter of well known Melbourne barrister William Ah Ket), Williams was taught initially by his father. At the age of twelve he went to Italy to study under Andrés Segovia. Later, he attended the Royal College of Music in London, studying piano because the school did not have a guitar department at the time. Upon graduation, he was offered the opportunity to create such a department. Being such a lover of the instrument, he seized the opportunity and ran it for the first two years. Williams has maintained links with the College (and with the Royal Northern College of Music in Manchester) ever since.

Williams is best known as a classical guitarist, but has explored many different musical traditions. He has collaborated with Julian Bream and Paco Peña and was a member of the fusion group Sky. He is also a composer and arranger.

Williams has commissioned guitar concertos from composers such as Stephen Dodgson, André Previn, Patrick Gowers, Richard Harvey and Steve Gray. He has also worked with composers from his native Australia, including Phillip Houghton, Peter Sculthorpe, Ross Edwards and Nigel Westlake, to produce guitar works that capture the spirit of his homeland.

He enjoyed a worldwide hit single with his recording of Cavatina by Stanley Myers, used as the theme tune to the Oscar-winning film The Deer Hunter (1979). The piece had originally been written for piano but at Williams' invitation, it was re-written for guitar and was expanded by Myers. After this transformation, it was used for another film, The Walking Stick (1970). In 1973, Cleo Laine wrote lyrics and recorded the song "He Was Beautiful" accompanied by John Williams. A year later, it was a top 5 UK hit single for Iris Williams (no relation).

At the invitation of producer Martin Lewis he created a highly acclaimed classical-rock fusion duet with celebrated rock guitarist Pete Townshend of The Who for Townshend's anthemic Won't Get Fooled Again for the 1979 Amnesty International benefit show The Secret Policeman's Ball. The duet was featured on the resulting album and the film version of the show - bringing Williams to the broader attention of the rock audience.

The relationship with Lewis led to Williams' classical-rock fusion band Sky being invited to give the first-ever rock concert to be held at Westminster Abbey - a benefit concert for Amnesty that Lewis produced in February 1981.

Willams has explained that he uses amplification for concerts in large halls.

Williams and his wife reside in London and Australia. He has a son, Charlie Williams, from his previous marriage to broadcaster Sue Cook.

John Williams was very helpful in bringing the works of Agustín Barrios back to popularity. Williams has often spoke well of Barrios' work, even stating the he believes Barrios is the greatest composer of guitar music.