Specs
Graphics Card
What are your hardware specs?
Try adding "newmenu 0" line to custom.cfg (or replacing value to 0 if it's already set to 1), might be because it's failing to open new ui.
Do 5.2x versions work?
Sorry for the incredibly late response, I have either been busy or at a friend's house. I checked my entire TB folder, the steam version, and there doesn't seem to be a "custom.cfg" file at all. 5.2x versions do not seem to work either.
newmenu 0
Also if Windows is hiding file extensions, it may appear as just "custom" in that same folder. Make sure that isn't what's going on.