If you have an EP6 engine then that has a wide band O2 sensor and, to my knowledge, you cant monitor the out put directly with a multi meter. If you don't have an EP6 engine let us know (I'm not really a Peugeot guy, LOL).
I would recommend that you get a cheap OBDII scanner (about $30 from EBay) and go to the "short term fuel trims" in the scanner's menu.
Then do as dmccurtayne says.
The short term fuel trim should sit around 0% (+- 5 or so %) at a stable idle and engine warmed up. The numbers might move around a bit but just take your best guess at an average. The fuel trim will go negative when you spray into the inlet, if the Oxy sensor is working. It will then go positive when you stop spraying, then stabilize at about 0 again.
If the trim does not move, or stays at about 20+% positive OR negative, then you MIGHT have a dead oxy sensor. There are other things that can cause that, but one thing at a time.
The cheap OBDII scanners have VERY limited functionality, BUT, they can be helpful for diagnosing engine faults. For the 30 odd bucks they are worth it (for engine faults ONLY).
Narrow band oxy sensors can be tested with a butane, propane or MAP gas torch and a multimeter or you can use a multimeter on the car.
Narrow band sensors only have 4 wires, Wide band sensors have 5 or more wires.
Too much information for a simple question?
Why do you want to test the oxy sensor?