There’s two ways to parse this question: what’s the relationship between computer science and academic psychology, and what’s the relationship between computer science and psychology in the sense of “how people think”.
To illustrate the difference, consider art and literature: nobody can deny that art and literature are deeply entwined with human psychology, and yet most insightful, high-impact art and literature is created by artists and authors who have little-to-no exposure to psychology as an academic discipline — and who certainly were not using psychological research methods to create their art!
So how does this apply to computer science?
Well, a significant part of computer science research is determined — to a larger or smaller degree — by psychology in the informal sense. Computer systems are ultimately used by, or at least run for, humans; systems have to be designed with this in mind, and the goals of research ultimately come down to human factors. Outside of research entirely contained in one area — theoretical work meant to further other theoretical work; systems research answering questions to improve other systems — you expect most projects to be at least guided by some sort of human factors.
Of course, often the “guide” is pretty generic: we designed this system to be faster because humans subconsciously perceive a response time >100ms as slow. That’s a real psychological insight, but it’s pretty widely known — and the rest of the research, the stuff that actually takes people time on the project, will all be based on improving performance. In a sense, yes, psychology mattered; in another, it really didn’t.
Even when the work is more closely tied to human thought, it doesn’t have to relate to academic psychology or its research methods (just like art!). Programming language design is a wonderful example: it is fundamentally a design discipline, and while it can try to use empirical methods like psychology, it doesn’t have to — any more than visual design does. It can, just like visual design can use AB testing — but the best examples of design rarely involve extensive empirical testing, neither in programming languages nor in visual design.
I also chose programming language design as an example because I think it’s far closer to art than people realize. Typography specifically is a great analog: nobody can deny it’s all about psychology and yet most people involved in typography don’t have formal psychology backgrounds or use psychology research methods. The same for CS.
There are only two fields that really rely on “formal” psychology: human-computer interaction (HCI) and software engineering. Both of these fields consistently rely on psychology research and use research methods developed by psychologists.