If there is no cabane, refuge or other civilized shelter nearby (or if you simply do not like them because you find them unfit for the natural feeling that surrounds vol bivouac), then look for natural shelter. If you have an interest in geology, or just a keen eye, you will be able to spot caves and crevices that might give you shelter, from quite a distance.
A real obvious one is shown on the right. Its entrance is roughly two metres above the ground and has a diameter of about one metre. Note something special? There is a metal rod sticking out of the rock, just near the entrance. Quite handy for accessing the cave using a rope. Without it, it is a bit tricky. Especially if you have a voluminous 20 kg on your back. The presence of the rod indicates that this is probably a 'civilised' cave: people have (often) been here before. The shelves on the entrance floor confirm this. It is probably safe to enter.
Of course, it is not always this easy to spot one. I was walking up Plateau de Cavillore one day for an evening soaring, when I encountered an elderly gentlemen dressed in an overall and hard head, carrying a torch and a rope. Together with his friends, he had been exploring a cave on (or rather in) the plateau. He had been exploring caves in France, especially in the region. His friends were a bit more fanatical than he was. The were still blasting their way down 400 m below the surface while the sun was setting.
Apparently, a lot of the (small) holes I had seen in the region were actually entrances to larger underground structures. Some of them were (ball room sized) chambers, others narrow shafts going down hundreds of metres. Maybe you would not like to live in those caves as they did 10.000 years ago, but some of them are quite a good home for the night. Especially when they have already been improved by other people with the same interest. Comfort is a relative issue here. The harder the rain, the more comfortable the cave. Even if it is a really uncomfortable one.
Depending on the type of rock and the sort of mountain you are on, you will find different types of caves. Some of those are really good for sheltering, others can be quite dangerous. As long as staying outside is less risky than seeking shelter, avoid caves that lack a solid structure. Caves in rocks consisting of a layered structure, such as shown by the image, can quickly become a headache if you are unlucky. Speaking from a geological time perspective, (parts of) the ceiling might come down any second here. From a human time perspective, nothing serious is probably going to happen in the next few hours or months. But this does not mean it won't happen soon. It is your choice. In order to perform your own risk assessment, click on the image to see the interior of the cave. It is dark inside (it is a cave remember), but you will be able to notice the floor is littered with (large) pieces of former ceiling. Some of them have come down quite recently.
You can usually guess the cave's solidity without exposing yourself to its (possibly) risky interior such as I have done here. For example, when you see lots of stones and rocks underneath yellow spots in an otherwise grey cliff, those stones have fallen down recently. The same could happen in the cosy cave that you found in this cliff. Having seen the loose rocks above the entrance here, I could have abstained from visiting the cave. However, I was curious and explored it anyway. As in most cases, curiosity cured rather than killed the cat here. Cured me from the thought of spending a long time in this cave, even though it had started raining outside.
Watch out with fires inside caves. The fire heats the rock, especially in the ceiling, expanding it locally. The only way it can expand is to where there is no other rock to keep it from expanding: out that is. A piece of rock that has been sitting there quietly for ages, could fall on your head as you cuddle up near your fire. It does not need to be cartoon-sized big to hurt you. A rock the size of a rugby ball has enough weight to finish you when falling on a critical body part.
I want a cave to shelter me from the elements and to be accessible without climbing gear, drilling tools, or dynamite. Most of all, I want to be able to get out after entering. On the aforementioned plateau, there is a cave called Trou du Mouton. It translates as Sheep Hole, but Sheep Trap is probably a better name. I have been told it is a hole in the ground (or in the ceiling, depending on your point of view) that provides access to a chamber underneath. Searching for a cave to shelter in, you do not want your last words to be "Found it!", and land on a heap of sheep bones just seconds later.
On the same plateau there is a prehistoric cave, or rather an 'Ancienne Habitation Troglodytique' if you are a real estate agent wanting to fetch a much better price than the current one. Its only access is a comfortable ledge that is a few metres wide. Unfortunately, it narrows to a foot wide for the last ten metres or so. The abyss of several hundreds metres deep right next to the ledge has always refrained me from taking these last few steps. I guess I am just too scared to do risky things.
There are also lots of human made caves, or rather tunnels, as well. In ancient times there was a lot more human activity going on in the mountains than just tourism and farming. There is still a chance that you might find caves or tunnels that use to serve as bunkers, chapels, stalls, shelter, water springs, or mines. For example, people extracted all kinds of natural resources from the mountains like coal, silver, iron, and precious stones. Did you ever wonder why there are villages called Argentière? Well, now you have the answer.
They have stopped those mining activities because it no longer pays (“Your ore ain't worth digging”), but the remains of those ancient times are often still visible. However, use those with care! They are no longer maintained. Things might collapse, bury, or otherwise hurt you.