Quotes below taken from:

GENERAL RELATIVITY FROM A to B
Robert Geroch
The University of Chicago Press CHICAGO & LONDON

c. 1978

**************************************

--- p. 12 ---

........

Similarly for "Physical space is three-dimensional." By the same
token, then, we are to regard space-time as four-dimensional, for the
location of a point in space-time (event) now requires the
specification of four numbers. One remark should be made in connection
with this four-dimensionality. The view has for some reason come to be
widely held that "the fourth dimension" is a deep and mysterious thing
which permits extraordinary happenings in the world, and which only a
few people can really understand. We emphasize that this is just not
true. We now already "have four dimensions." On the other hand, we
have not yet introduced a single statement about the way the physical
world operates that was not known to all of us since childhood. True,
we have perhaps been more careful and precise in our discussion than
we might have been previously, yet the fact remains that, with no
additional contributions whatever to our basic fund of physical
information, we have arrived at a description in terms of four
dimensions. If you like, "four dimensions" is just a convenient way of
describing the world and thinking about the world, nothing more. Is
the "fourth dimension" real? It should now be clear, from these
remarks and from the discussion of "reality" in chapter 1, that
physics will not answer such a question, and that the attitude of
physicists will be that such a question is not germane. There is the
physical world, and then there is our description of it. As long as
our description is reasonably clear and reasonably accurate, there
will be no objections. We can change our description every Friday
morning if we wish. Nature doesn't care about our descriptions; She
just keeps rolling along. If these days we choose to describe Her in
terms of a space-time, and if that space-time has four dimensions,
then, as long as that description is reasonably clear and reasonably
accurate, that's fine and that's the end of it. Tomorrow's description
may have two dimensions or nineteen dimensions. All of us, I can
assure you, now understand "the fourth dimension" as well as anybody.

**************************

Just a brief followup from me. I would myself have rather said: There
is the physical world, there is our collection of human sense
impressions of it, and then there is our description of physical world
as a free creation and as somewhat consistent with our sense
impressions. (We have the right to claim that some sense impressions
are illusionary or meaningless. We also have the right to model freely
with invisible madeup things (physical concepts), if we so choose to
do.)

Physics is the search for the smallest set of rules which completely
describes the behavior of the inanimate material realm under natural
conditions. Rules come out of theories, but theories come out of the
creative human mind.

Patrick