In August 1964 Joe McPhillips and I walked into the Parade Bar in Tangier to find Bill Burroughs. Standing beside him was a tall man sipping his favourite cocktail – a bourbon mist with a twist. He studied us with a pale blue gaze.

This turned out to be Burroughs’s close friend and collaborator Brion Gysin, the poet, painter of the Sahara and inventor of the Dreamachine, who had introduced Bill to the ‘cut-up method’.

Brion had recently come back to Tangier to begin a novel, The Process. He had moved into a one-room flat in the Inmueble Itesa, a featureless concrete apartment block where Paul and Jane Bowles lived. Paul made his headquarters on the fourth floor, above Jane. Neither Paul nor Brion had a telephone. Paul and Jane communicated via toy telephones connected by a wire that dangled from the window.

A few months after Brion was born, his father was killed at the Somme. The boy was raised in Canada by his grief-stricken mother: the wide open spaces of the Canadian prairie later influenced his illusionist desert landscapes, which Paul Bowles, who captured that vast emptiness in The Sheltering Sky, declared to be the only true paintings of the Sahara. At sixteen he was shipped off to Downside. After failing to win a scholarship to Oxford, he set out to ‘have adventures and see visions’. As John Geiger explains in his meticulous biography Nothing Is True, Everything Is Permitted: The Life of Brion Gysin, the glorious adventure was about to begin.

He explored the hidden world of Paris with ‘all those dark underworld passageways of queer sex’. He met Marie-Berthe Aurenche, Max Ernst’s wife, and was introduced into Surrealist circles. In 1935, not yet twenty, he was invited to contribute works to a show of Surrealist drawings at the Aux Quatre Chemins gallery that included pieces by Ernst, Dalí, Magritte, and Picasso. What should have been a marvellous opportunity proved a bitter disappointment. On the afternoon of the opening he found Paul Eluard taking his pictures off the wall. ‘What’s happening?’ Gysin demanded. ‘Orders from Breton, you’ve been expelled.’ The expulsion had the effect of a curse, and he never trusted the art world again.

For three years he wandered around Europe. Finally, at the age of twenty-three, he was awarded an exhibition at the same gallery. This time it was a solo show, and he was proclaimed ‘the most promising painter of his generation’.

When war broke out he was drafted into the Canadian Army and assigned to the Japanese language school in Vancouver, where he studied Japanese script: ‘It was enough to influence my whole life as a painter.’ At the same time he turned out a book, To Master: A Long Goodnight, based on a story told him by the great-grandson of the original Uncle Tom of Uncle Tom’s Cabin. It was published in 1946 and won him a Fulbright Fellowship.

Returning to Europe, he met Jane Bowles. Paul Bowles, already an acclaimed composer and author of the bestselling The Sheltering Sky, invited him to Tangier. Gysin felt immediately at home and went ‘native with a vengeance’. He was to live in the city on and off for twenty years. Through the Moroccan painter Hamri he discovered the village of Jajouka, where the rites of Pan still flourished. In Tangier he opened the 1001 Nights, an exotic restaurant. One of his culinary delights, hashish fudge, merited an entry in his friend Alice B Toklas’s cookbook.

Back in Paris in 1958, Gysin and Burroughs teamed up at the Beat Hotel at 9 Rue Gît-le-Coeur. The four years they then spent together became the most intensely creative period of Gysin’s life. He convinced Burroughs of the need to use painters’ techniques in writing, arguing that ‘writing is fifty years behind painting’. What he had in mind was the controversial ‘cut-up method’. Bowles expressed doubts about this, claiming that ‘anyone who came under Gysin’s influence fell ten years behind in his career’.

The best time to visit Brion or Paul in Tangier was late at night. As the door of the apartment block was locked at nine o’clock, the only way to gain entry was to wake up the night watchman, who slept on a pile of sheepskins in the garage. For a dirham he would open up, always with a smile.

Targuisti, Brion’s friend, cook and general factotum, was usually in attendance, as was the gentle Salah, Brion’s lover, a tall moustachioed black man. Save for a double bed that took up half the space, there was no furniture in Brion’s flat, only a couple of poufs and a taifor, a low round table that was rolled out at mealtimes. Brion had all his meals Moroccan-style, sitting on the floor and eating with his fingers. In an alcove near the window he had rigged up a desk. The walls were hung with his stunning desert scenes and calligraphic works.

The household language was Spanish. Moroccan mint tea and shira, Brion’s preferred smoke, were always available. He sat cross-legged on the bed, tall, elegant and guru-like, reeking of patchouli, sebsi in hand, mesmerising his guests with a non-stop flow of ideas.

As Virginia Spencer Carr notes in the first line of Paul Bowles: A Life, Bowles grew up hating his father. During the years of being an enfant terrible he restlessly floated around Europe. Carr gives us a fantastic tale of literary and musical creation, hectic non-stop travel, work accomplished in desert and jungle and on board ship. She chases this elusive artist through Africa, Central America and the Far East, finally catching up with him in Tangier.

The entrance to Paul’s flat was a tiny hall stacked with suitcases plastered with hotel and steamship stickers. There was a wall of books, low banquettes, and a trunk with a rounded top. In an atmosphere poised between night and day, Paul, dressed in casual but conservative American fashion, poured tea and doled out cookies. A fire flickered in the hearth. Usually in attendance was the Moroccan storyteller Mohamed Mrabet, sitting naked to the waist chopping kif with a hunting knife. His powerful muscularity and moody nature suggested a volcano about to explode, especially if your visit interrupted a recording session. Again, the household language was Spanish.

During those late-night sessions Paul was an attentive host and deftly anticipated the effects of the weed he loved to smoke. When you got the munchies he produced a bowl of fruit. When your throat began to feel like the inside of a tin cup, he served Lapsang Souchong. When there was a gap, something missing (you were too spaced out to know what it was), he lit a joss stick dipped in some magic ointment brought back from the East. When the room lapsed into silence, you were aroused by the sound of bells, tiny bells that once tinkled delicately in a Thai temple.

Jane liked to cook. Their spontaneous affection and sense of fun made them seem more like brother and sister than man and wife. Her speciality was jugged hare in a red wine sauce. Sherifa, Jane’s Moroccan companion, rattled on in Arabic in her gruff mannish voice, a rough alien presence who acted like she owned the place. Jane seemed like a fragile figure that had been knocked to the floor: the pieces had been glued back together, but crudely, and the cracks showed.

In his biography of Gysin, Geiger fails to mention his contributions to the American School of Tangier. Headmaster McPhillips had a lot of local talent to draw on for his annual theatrical productions: Paul Bowles to write the scores, Yves Saint-Laurent to design the costumes, and David Herbert to ensure that le tout Tanger was in attendance.

Brion agreed to paint the backdrop for Paul’s play The Garden. McPhillips and I stretched a huge canvas – about fifteen feet high by thirty long – and propped it against the backboard on the basketball court at the American School. We bought the pigments he requested: earth colours – yellow ochre, terracotta, brown, grey and green. We mixed the paints in buckets and arranged them in a row before the canvas. Brion had requested not paintbrushes but sticks; each bucket contained the bamboo stick we had mixed the paint with.

At the appointed hour Brion himself arrived. He sat down in front of the canvas and lit his sebsi. He pulled from his pocket a house painter’s brush, stuck it in the brown paint and like a bullfighter advanced on the canvas. He made two sweeping arcs that looked like breasts, then one long phallus-like slash between them. This was supposed to be a desert scene. We wondered what he’d do next.

Then he began to work in earnest. Darting along the line of buckets, he picked up the sticks and started flinging the paint onto the canvas. Quickly, he developed a technique: he directed the paint in globs or spattered the canvas with a fine rain of colour. Before our eyes the desert appeared. Those two mounds were not breasts but sand dunes of the Sahara. With a flourish of green on top, that phallic stab of paint in the middle turned into a palm tree.

In a matter of minutes he had covered the entire canvas with paint. There was the beleaguered palm tree of Paul’s garden. We had witnessed the creation of an oasis. We marvelled at the ease, the grace, the sheer athleticism with which he had brought the desert to life. The technique was carefree, but the result evoked the marching dunes that must inevitably swallow the oasis.

The names of Brion Gysin, Bill Burroughs and Paul Bowles will always be linked to Tangier because they did their best work there. Brion and Bill looked up to Paul because he was the first to arrive and the first to achieve artistic success. Bill’s body lies in Kansas. Paul’s ashes are in a family cemetery in New York State. Brion’s were scattered to the winds from the Cave of Hercules outside Tangier.