I use FRAPS but it makes the files huge, 10min of recording is around 9GB, (1366x768 30-50fps), I usually pass the video through an editor (Premiere Pro) and I get a 500MB file without cutting anything and the same quality (720p)

Suggesting OBS isn't enough because you may want to use Nvidia NVENC or AMD VCE for h.264 encoding. If you have a recent card that supports either, you can use the video encoding hardware on the card without affecting graphics or CPU performance.

OBS has NVENC support built in, but you need a need a special branch of OBS that supports VCE if you have a compatible AMD card.

[ Register or Signin to view external links. ]

CPU encoding will always look better for the bitrate, but the performance hit is probably not be worth it. I use above 20 Mb/s with VCE and it looks nice enough for 1080p60.