I procured a VPS solution from a hosting provider on behalf of my client, specifying a modest configuration comprising a dual 3.6 GHz CPU setup, 6 GB of RAM, an ISP control panel, and comprehensive administrative privileges.

Subsequently, I requested a few tweaks to optimize performance - migrating Apache to mpm-itk and deploying a couple of CLI tools (which, being non-daemonized, only consume system resources during active usage).

However, upon reviewing the post-configuration metrics, I observed a significant spike in memory utilization, from 20% to over 40%. Notably, I had yet to establish a connection to the server or control panel, meaning no websites or additional settings had been configured.

This unexpected surge in resource consumption raised some red flags, but the hosting provider seems unconcerned, dismissing my inquiries without investigating the underlying cause (despite our agreement including full administrative access).

Can anyone weigh in on this - am I being overly demanding, or was it reasonable to expect a more thorough examination of the issue?

I've dealt with my fair share of shady providers, and this behavior screams "we don't care about our customers." The fact that they're not even willing to investigate the issue suggests that they're either incompetent or hiding something.
You should be prepared to take your business (and your client's) elsewhere, as this level of disregard for customer concerns is unacceptable.

Delve into the root cause of the memory consumption spike. It's possible that a rogue system process or a resource-intensive application is hogging the memory. Alternatively, it could be a case of a Distributed Denial-of-Service (DDoS) attack targeting the SSH server, bypassing the firewall and causing the increased memory utilization. This scenario is not uncommon, especially when dealing with brute-force attacks that can lead to a significant surge in resource consumption.

Consider a scenario where server-side operations are executed, consuming memory resources. However, it's essential to note that memory deallocation isn't always instantaneous. Modern operating systems employ memory management optimization techniques to minimize unnecessary operations, thereby deferring memory release until it's genuinely required.

In the interim, unused memory may be relegated to a 'cache thrashing' state, awaiting release in a queue. Only when the system frequently resorts to swap space, exceeding reasonable thresholds, should concerns arise; otherwise, there's no need to be overly vigilant.