Great that you wish to get started! Spending time outdoors is a free but wonderful joy.
One piece of advice that I would give is to not rush and buy a lot of gear, gear is fun and buying it can be rewarding. But the main attractiveness with bushcraft for most people is to actually spend time outdoors.
If you have not spent any time outdoors and want to try it out I would recommend just taking a day trip where you spend a few hours outdoors when the weather is good, this reduces the amount of gear you need to a minimum and you can just enjoy spending time outdoors.
The absolute minimum that you would need is:
- Clothes suitable for the weather
- Food that does not require cooking
- Insect repellant
- Small backpack to carry your stuff (guess a bag could work, even though it's not that convenient to carry)
- Bag to put trash in
- A location worth spending some time at
The cost of the minimum gear you need will probably not exceed $10 + transportation since you hopefully already have a backpack and clothes, what you need to buy is some food and insect repellant.
If you wish to add things I would recommend:
- Coffee, tea or hot chocolate
- Basic fire starter tools (cigarette lighter or matches)
- Fire crafting skills (most important is to know how to put out the fire safely to prevent wildfire!)
- Food that require cooking
- Basic first aid kit
- Basic cooking and dining equipment suitable for an open fire (metal pot or similar)
- Knife suitable for wood (Mora Companion is a very good and cheap knife for $14 + shipping)
The extra gear would add probably around $30 more, but can be worth it to improve the bushcraft experience some.
Selecting a day
Look at the weather reports, try to select a day that will have convenient weather for you to be outdoors in. Which means that it shouldn't be rain or storm and the temperature should be in a good range.
Things bushcrafters wish they knew early on
- Gear does not make up for knowledge. You can bring your whole house into the wild, but if you don't know how to use the tools it's useless. Build your kit slowly. 
- Bushcrafting is not about buying expensive gear and avoid falling for YouTubers and internet that try to convince you that you need to buy stuff.