About six years ago the first section of the now celebrated High Line was opened in New York and made a palpable hit both locally and internationally. Locally it revealed what one might have guessed, that the inhabitants of Manhattan’s downtown suffered a severe lack of amenity. Every place to walk or run or ride a bike, every place to exercise the dog, is valuable and well used. This new and unusual park, restoring and converting the tracks of a disused overhead railway, was reserved neither for running nor biking nor walking the dog, but rather for strolling, sitting and sunbathing, and for the novelty of looking in on buildings old and new, from unusual angles and with an unusual degree of impertinence. That is, it soon became something of a place for voyeurs and exhibitionists — men with large chests, for instance.

But it also, to an incomprehensible degree, attracted international tourists — five million visitors a year by now to an ugly part of the city that used to house the meatpacking district. It seems always crowded, like the passeggiata of a small Italian town, but whereas I think I understand the spirit of the passeggiata — one saunters forth in the cool of the evening, to see and be seen by one’s fellow citizens — I do not understand what the tourists at the High Line expect. To be seen by fellow tourists? To hook up with manly mesomorphs? I don’t get it. Certainly nobody seems to be paying attention to the plants.

New Yorkers, who tend to be rather vague about the subject of gardening, sometimes say that the planting of the High Line is essentially an ingenious collection of weeds. In fact it is kept meticulously weed-free. They might, at a more sophisticated level, say that it is a collection of native plants. In fact, though it is rich in native plants, it is highly reliant on garden varieties. Study the official plant-list and you will see that there is no section for annuals — plants that survive by seeding themselves within a single season. To create an ecology without annuals strikes me as perverse. But it is not the only perverse thing about Piet Oudolf’s celebrated design.

There are no lilies. There’s only one kind of iris, one kind of poppy. There are very few plants that show a strong clear blue. There are seven kinds of rose out of a great number of possibilities, and one kind of honeysuckle. The Oudolf style loves muddy purples and browns and all the limited things that grasses can do. It constitutes a Dutch/German version of ‘ghastly good taste’. It is all about exclusion and a timidity of the palette. It’s not by any means the worst planting in New York (Park Avenue with its massed tulips and Kanzan cherries takes that award each year) but it is very far from the best. If you want to see native plants grown beautifully, the New York Botanical Garden is far, far superior.

As the High Line grew from a charming idea into a monster, pushing up local property values and withdrawing the offer of amenity it had once held forth to the local population (it is just too crowded), at the same time the Whitney Museum was going ahead with a policy of expansion. It would build its new museum on Gansevoort Street, at the southern end of the High Line, and move from its too-small premises on Madison Avenue. It would not — it could not —sell the old Whitney. But it would lease it for eight years to the Metropolitan Museum, which would use it to display its 20th-century and modern works.

This all sounds very co-operative. It is not very co-operative. It is very competitive. You always have to remember that whereas cities like London, Paris and Berlin have (to varying degrees) a co-ordinated policy for their great museums and galleries, which are state-run, New York has no such arrangement. The Met, the Museum of Modern Art and the Whitney all compete: they all collect in areas that overlap, and they are all out to woo the same sort of patrons, and the same visitors. The Whitney was founded to collect American art, after (as has been much mentioned in the press recently) the Met turned down, in 1929, the offer of 500 American works from the collection of Gertrude Vanderbilt Whitney. But the Met has what the Whitney no longer has (because it divested itself of its pre-20th-century art), a pre-eminent collection of American art of previous centuries, and superb holdings in the American decorative arts. And the Met collects 20th-century art in direct competition with MoMA.

What the Met has lacked hitherto is good gallery space. The Lila Acheson Wallace Wing (or LAW, as it is known) is not well designed for the display of works of art, and it feels, anyway, like something of an afterthought, tucked away at the back of the great building. The leasing of the old Whitney Museum building gives the Met the chance to pull down LAW, remove an underground carpark, enlarge and rebuild its modern galleries. To this end, it has hired David Chipperfield, who did such striking work at the Neues Museum in Berlin, as architect of the new scheme.

So it is that when you fight your way down to the southern tip of the High Line and see the new Whitney, what you are seeing (the Renzo Piano building that has been much compared to a photocopier) is certainly, in the medium term, very good news for the Met. Is it good news for the Whitney?

Renzo Piano’s calling card is the Menil Collection in Houston, Texas — an intimate and friendly structure that everyone has always loved. It opened in 1987. Less happy was his recent work at New York’s Morgan Library, where he was given a second shot at providing modern ancillary spaces serving the grand old structure. Well, the term ancillary is mine — it may not have been in his brief, which may well have been a bad brief. There is something almost offensive about the way you move from the new atrium to the old library, as if the original building were a tolerated appendage to the new.

The new Whitney could hardly have been given a better site, and the building takes advantage of its setting. From the inside it piques the new visitor’s curiosity, offering views of the Hudson and its ramshackle, post-industrial neighbourhood. It is from the outside that it appears a touch insipid. As a home for the Whitney Biennials it seems to me it will serve extremely well, being in the right part of town, and offering large and flexible spaces.

The opening show, which is drawn from the museum’s permanent collection, had a curious effect. It seemed to smooth over the transitions between star performers and minor artists. American art is rich in interesting minor talents, people who in the course of the last century responded to such themes as the Dignity of Agriculture, or the Life of the Modern City, and produced work that is not only charming but also possesses a great illustrative historical interest. It must be quite a problem — it must be one of the Whitney’s recurring problems — to present this absorbing but vulnerable work alongside that of the more clamorous talents and acknowledged stars.

Whether it was a mere effect of the weather, or of a judgment distracted by that inevitable sense of occasion as a museum opens its doors for the first time, I felt that the minor pieces were sometimes trying to bring down the greater. The title of the show: America Is Hard To See. Well yes, it made its point.