At continuous phase transitions, quantum many-body systems exhibit complex, emergent behavior. Most notably, at a quantum critical point, correlations decay as a power law, with exponents determined by a set of universal scaling dimensions. Experimentally probing such power law correlations is extremely challenging, owing to the interplay between decoherence, the vanishing energy gap, and boundary effects. In this work, we used a Rydberg quantum simulator to adiabatically prepare critical ground states of both a one-dimensional ring and a two-dimensional square lattice. By accounting for and tuning the openness of our quantum system, which is well-captured by a single phenomenological length scale, we directly observed power law correlations and extracted the corresponding scaling dimensions. Our work complements recent studies of quantum criticality that use the Kibble-Zurek mechanism and digital quantum circuits.