However, you would have had still colonies in the filter, which would have been great. The point VM now makes is without a source of ammonia your bacteria will starve, very quickly. By the time you added your betta, it seems, your bacterial colonies had died off and he may have suffered ammonia burns. It would have been better to put him into the tank very shortly after you had rehomed your goldfish.
Before putting any new fish into the tank, just make sure you get yourself a proper testing kit and cycle your filter properly, so it can cope with any new fish. Alternatively, if you know someone locally with a fish tank, ask them for gunk from their filter so you can 'seed' yours (basically kickstart the different bacterial colonies) but make sure you feed them a source of ammonia.
We all make mistakes sometimes or think we're doing something useful and then things go wrong. All of us have been there. Hope you get it sorted now
