Using variable with public const modifier

I'd like to have a variable in my class that is const when accessed from public scope and non-const when accessed from a private or protected scope. I'd like to avoid having a getter-method that returns a const reference to the variable. I tried messing around with the using keyword, but no success so far. Ideally it would look something like this:

class Foo
{
public:
    using var = const m_var; // This is of course gibberish
private:
    int m_var;
}

I could add a public const reference member variable and let it point to the private variable, but that way my class would get 4 bytes bigger, which I would like to avoid. Would a compiler realize that and optimize out that reference?

Any other ideas on how to solve the problem?