With just days to go, Cape Town's sleek new Green Point stadium is in the final stages of preparation for the World Cup. This stunning white apparition rises like a porcelain bowl from a podium set in restored parkland, between the breakers of the Atlantic and the commanding backdrop of Lion's Rock, Devil's Peak and Table Mountain. Is there a stadium anywhere in the world with a more sublime setting?

Fifteen storeys high with curving walls, the new "Diva of Cape Town" is sheathed in a translucent fabric mesh, made from woven fibreglass coated with Teflon. This sounds coldly technical, but during the course of the day (and depending on the weather), the building's filigree skin glows blue at noon, rose in the late afternoon and red at sunset. On stormy days, it can appear silver, grey or even pewter. By night, and with its 360 internal lamps switched on, the skin becomes all but transparent, revealing the undulating swoops of the stadium's three tiers of seats, its airy public concourses, and something of its quietly thrilling structure.

But there has been criticism of Green Point – and in particular of its location, in an opulent part of town a long way from the townships where most football fans live. (To put it into some context, a bottle of Bollinger La Grande Année, 1999, £275 a pop in a swanky nearby restaurant is slightly more than the cost of employing a labourer for a month on a building site.) An enormous amount of public money has been spent on this and other South African football stadiums: are they primarily a branding exercise, proving to the world that South Africa is a go-ahead, modern nation? How well do they serve local people, who in many cases will have to trek a long way to reach them?

Green Point is one of 10 stadiums hosting the World Cup, five of them existing buildings recently brought up to date, five of them brand new. Three of these have been designed by a consortium of architects led by GMP Architekten, Cologne: Green Point (with a group of Cape Town practices); the Wembley-like Moses Mabhida stadium, Durban; and the elegant Nelson Mandela Bay stadium in Port Elizabeth (again, with a consortium of South African practices). Each is as crisply refined as you might expect of the very best German engineering and design: these stadiums are as pristine as a Braun radio by Dieter Rams, or a Silver Arrow Mercedes-Benz racing car.

The thinking is that all the necessary colour will be provided by players, crowds and the human drama enacted in these exquisite sports machines. There appears to have been no attempt to play to the gallery – no sentimental rainbows or kaleidoscopes of colours – although I hear that some politicians wanted the fabric mesh of the stadium walls finished in a leopard-skin effect.

In recent decades, architects and engineers have produced a number of superb new stadiums. Munich's Allianz Arena, Milan's San Siro, Bari (by Renzo Piano), Wembley (Norman Foster) and Berlin's beautifully remodelled Olympiastadion show how far inspiring design can enhance the beautiful game. A great stadium offers uninterrupted sight lines, a compact arena so that players and crowd feel close to one another, and easy entrances and exits. They work much like great opera houses: the layout of the building creates a sense of compression and release, heightening the sense of drama and occasion: intense in the arena, relaxed in the public spaces around it. And if they can offer that something extra – a memorable facade, a dramatic roofline, then they become part of the drama they underscore.

It is, of course, perfectly possible to have an electric atmosphere without memorable architecture: Manchester United's stamping ground at Old Trafford is based on the school of out-of-town supermarket design, and it barely matters. Location, too, can be as important as design: look what happened when Juventus moved from their battered yet much loved inner-city home – the Stadio Municipale, Turin – to the architecturally impressive but soulless and windswept Stadio delle Alpi, way out of town. The intense atmosphere of the old inner-city games was lost.

In South Africa, Green Point is certainly an emotional touchstone. It used to be home to an 18,000-seat stadium dating from the 1940s, hosting acts such as Michael Jackson, Paul Simon and U2; this historic, if modest, building was demolished in 2007. The structure of the new 4.4bn rand (£40m) stadium, financed by national, Western Cape and city governments, was completed last December ahead of time, with up to 2,500 workers on site at any one time.

530 toilets, 16 lifts and a jail

Occupying a site the size of six city blocks, the stadium is enormous, with a delicacy and grace that belies its bulk. The arena rises from a tautly designed concrete and glass podium, housing a car park, club rooms, offices, a medical centre, a press centre and a police station with cells. Above are the three tiers of seats, along with cafes, kitchens, shops and 250 VIP lounges, all leading off spacious concourses and plazas. Somewhere in the structural mix, there are 59 gates, 115 entry turnstiles, 530 lavatories, 16 lifts and four TV studios.

All this is impressive, but it is the view from the 68,000 seats that is truly spectacular. Above, daylight filters through the fabric canopy, set beneath a circular roof made up of 9,000 glass panels. The very centre of the roof – the space above the laser-levelled rye grass pitch – is open to the elements. This design, as well as the curvature of the walls, is intended to hold the most intense sunlight at bay (and harsh Atlantic storms); it should also contain noise as far as possible within the confines of the stadium. (The stadium is set close by a hushed and wealthy residential neighbourhood. For the same reason, lighting has been largely contained within the stadium, avoiding the need for the usual powerful floodlights mounted on giant external gantries).

A huge effort has been made to ensure that this vast building is a pleasure to look at from any angle. The roof in particular is beautifully resolved, especially when seen from above – from Cape Town's hillside suburbs, the top of Table Mountain and, of course, the television helicopters covering the football. As Robert Hormes, GMP's project architect, says: "The roof is the fifth facade of the building." He admits that it presented a complex design problem: his aim was to build the stadium as low as possible, to help withstand strong ocean winds and to protect views, while shaping a structure containing as many seats as possible. The result is a roof designed "like the rim of a bicycle wheel", holding all the forces at work within a lightweight and delicate-looking frame.

Not everyone wants to like Green Point, or indeed any of the other new stadiums. Fahrenheit 2010, a recent documentary by Australian film-maker Craig Tanner, has been highly critical of Green Point and its siblings. Tanner believes the entire project has been a waste of money that will lead South Africa into serious debt. The late Dennis Brutus, the veteran anti-apartheid campaigner jailed with Nelson Mandela on Robben Island and one of Tanner's interviewees, argues on screen that: "When you build enormous stadiums, you [are] shifting resources . . . from building schools or hospitals and then you have these huge structures standing empty. They become white elephants." This is an age-old concern, reinforced by the questionable legacy of ambitious sporting venues built for World Cups and Olympics over the last 60 years. Struggling with its economy and yet attempting to woo 450,000 foreign tourists this summer, South Africa hopes its new stadiums will make the leap to first-rate venues for sport, music and other mass gatherings when the Fifa circus has moved on.

The first game to be played here will be between Uruguay and France on 11 June; England will play Algeria at Green Point a week later. And after this summer? This is the 4bn-rand question. The stadium will be run by SAIL/Stade de France, a Parisian firm, and should be home to local football teams Hellenic, Santos and Ajax Cape Town. Its special (albeit controversial) location, intelligent design and civilised presence should attract other users – though there are no guarantees. Will Green Point stand the test of time and qualify as one of the world's great football stadiums? We will have to wait until next month to see.