How universal is human conceptual structure? The way concepts are organised in the human brain may reflect distinct features of cultural, historical, and environmental background in addition to properties universal to human cognition. Semantics, or meaning expressed through language, provides direct access to the underlying conceptual structure, but meaning is notoriously difficult to measure, let alone parameterise. Using cross-linguistic dictionaries, we provide here an empirical measure of semantic proximity between concepts and analyse the structure of a network derived from it. Across languages carefully selected from a phylogenetically and geographically stratified sample of genera, translations of words reveal cases where a particular language uses a single polysemous word to express concepts represented by distinct words in another. We us the frequency of polysemy linking two concepts as a measure of their semantic proximity, and represent the pattern of such linkages by a weighted network. This network is highly uneven and fragmented: certain concepts are far more prone to polysemy than others, and there emerge naturally interpretable clusters that are loosely connected to each other. Furthermore, the networks of different language groups exhibit consistent structures, largely independent of geography, environment, and literacy. We therefore conclude the conceptual structure connecting basic vocabulary studied is primarily due to universal features of human cognition and language use.