The complex spatial structure evident in spiral galaxies is understood either in terms of instabilities of quasi-stationary states, or a result of dissipative non-gravitational interactions. We illustrate here, using numerical simulations, that purely self-gravitating systems evolving from quite simple initial configurations can give rise easily to structures of this kind of which the lifetime can be large compared to the dynamical characteristic time, but short compared to the collisional relaxation time scale. More specifically, for a broad range of non-spherical and non-uniform rotating initial conditions, gravitational relaxation gives rise quite generically to long-lived non-stationary structures of a rich variety, characterized by spiral-like arms, bars and even ring-like structure in special cases. These structures are a feature of the intrinsically out-of-equilibrium nature of the system's collapse, associated with a part of the system's mass while the bulk is well virialized. They are characterized by predominantly radial motions in their outermost parts, but also incorporate an extended flattened region which rotates coherently about a well virialized core of triaxial shape with an approximately isotropic velocity dispersion. We briefly discuss the possible relevance of these simple toy models to the observed structure of real galaxies. On the one hand, we discuss the determination of the velocity fields of THINGS galaxies showing that the unambiguous identification of radial motions in 2d maps is possible only if the galaxy is a disc. On the other hand we consider recent observations in our Galaxy (APOGEE and GAIA) that show that the galactic disk seems not to be a stationary axisymmetric system. We conclude by stressing that these simple models illustrate the possibility that the observed apparent motions of spiral galaxies might be explained by non-trivial non-stationary mass and velocity distributions without invoking a large amount of dark matter halo or a modification of Newtonian gravity.