This error seems bizarre especially when you are actually trying to increase the guest RAM and the error is about disk space. The cause could be the following:
- By default, ESX will configure a swap on the VM's datastore equal to the RAM. So, when RAM is increased, more disk space is required. The fix is to reserve memory in resource allocation settings for the VM.
- There really isn't enough disk space on the datastore. Reduce Hard Disk size for the VM.