I can store a container of indices that are invalid,
Or you can store a container of indices that are valid. Unless you are on a very constrained platform, an extra 4M (assuming 1 million elements and 4-byte indices) for that container just isn't all that much space.
and get max and min,
Why not just store their indices when you build the vector?