array_values() doesn't work with ArrayAccess object.
neither does array_keys()
why?
if I can access $object['key'] I should be able to do all kind of array operations
No, you've misunderstood the utility of ArrayAccess. It isn't just a sort of wrapper for an array. Yes, the standard example for implementing it uses a private $array variable whose functionality is wrapped by the class, but that isn't a particularly useful one. Often, you may as well just use an array.
One good example of ArrayAccess is when the script doesn't know what variables are available.
As a fairly silly example, imagine an object that worked with a remote server. Resources on that server can be read, updated and deleted using an API across a network. A programmer decides they want to wrap that functionality with array-like syntax, so $foo['bar'] = 'foobar' sets the bar resource on that server to foobar and echo $foo['bar'] retrieves it. The script has no way of finding out what keys or values are present without trying all possible values.
So ArrayAccess allows the use of array syntax for setting, updating, retrieving or deleting from an object with array-like syntax: no more, no less.
Another interface, Countable, allows the use of count(). You could use both interfaces on the same class. Ideally, there would be more such interfaces, perhaps including those that can do array_values or array_keys, but currently they don't exist.