Downtown Toronto becoming like ‘Downton Abbey’ as service workers get pushed farther from the core