So I did switched to pipewire with pipewire-pulse (Archlinux) on my main workstation.
So far, I finally have no more frequency bugs. With pulse, sometimes the frequency would shift randomly when starting streams (especially when joining a discord call while music was playing) since the switch I never had any frequency bugs. I'm using a Focusrite 2i2 as my sound card.
Some bugs I noticed: when switching my main device using the KDE sound indicator, sometimes some applications don't switch. Switching device usually is very smooth, but sometimes don't happen.
It may not be related, but rarely I have a considerable delay when talking on discord, but only happened twice since in two months and was resolved each time by restarting discord. It may be my old 4790k sweating because I'm compiling stuff often and on all cores.
I have a huge thank you for everyone in this community and also everyone that made pipewire happen. It's already in a pretty good shape and simply removed most of the problems I had with pulse.