As a tourist in Paris, you obviously plan (even if you are on a budget) to go shopping in the best French shops. You'll go to the Galleries Lafayette, to the Printemps, to the Champs Elysées, to Avenue Montaigne, even if it's just to admire the displays. To see something different from all that, I would strongly recommend you to go to the classic Parisian grand magasin: Le Bon Marché.
Bon marché literally means "cheap", which is not really the case at the moment. Indeed, they sell high-end brands like Valentino, Kenzo, Gucci etc, so not the usual retail brands. But this shop in two buildings doesn't only offers clothes. They literally sell everything: home products, furniture, high-tech, menswear, perfumes, makeup, gadgets, toys, children's clothes etc. Take a look at their lampshades collection, as well as all the wonderful stuff they have for cooking.
You can still manage to get a souvenir from The Bon Marché, as the little items are quite cheap. I once got a beautiful flamingo pen for only one euro. Just take a good look around and be curious.
They even have a whole building only for food: La Grande Epicerie. This part is maybe one of my favorite shops in Paris. You can treat yourself to exotic delicious food, but as it is a supermarket you don't spend as much as you would in a restaurant. Their pastries are actually pretty much the same price as everywhere, but they are better. It is also an ideal place to bring back souvenirs for your friends: who would refuse quality French food?
Finally, the best thing about the Bon Marché is its atmosphere. I genuinely go there only to cruise around. Just like in Breakfast at Tiffany's, you feel like nothing bad could happen here. The regulars from the neighborhood are there to do their usual overpriced shopping and you feel like observing a completely different world. The building itself is beautiful and really well conserved, and the displays are outstanding. Go, hang around there during a week day for peaceful shopping.