Wow, that's a lot of questions. I shall try to answer as many as possible.
I did my Masters in India and PhD in Europe (not the UK). In most countries in Continental Europe, a PhD student is considered a full staff member of the laboratory and is paid according to fixed norms, as the research grant includes the PhD student salary. The salary does not allow one to have a luxurious lifestyle but is quite sufficient. In fact, I knew quite a few people who could maintain their spouses as well, and some had kids too.
Whether you need a PhD for a research role really depends on the environment you choose to work in. Usually, academic labs require a PhD for any sort of role that involves project management or leadership, with non-PhDs being hired only in supporting roles. It is quite different in industrial R&D units though. I know several non-PhDs with Masters degrees, who have risen up to the Senior or Staff Scientist level solely on the basis of their experience in the industry. Many company research directors too have no PhD, and in fact, have PhDs reporting to them.
As for what happens after a PhD, the scenario can vary a lot. Some of the extremely nerdy types prefer to continue doing research in an academic setting. These people end up becoming group leaders or professors if they do some good work and get some high impact publications. Some join industrial R&D labs. Most others get frustrated and end up doing a variety of other things. Some do an MBA and enter the industry, others take up teaching or a variety of other professions. Sadly, a few have even ended up driving taxis to make ends meet. However, there are very few in the last category and most PhDs usually do well in life.
Research projects in academia are long-term and can run for years. Short term hires are usually recruited to work on a small aspect of the big picture. In the industry, the goals are more focussed and short-term in nature. As for taking up new projects, people do that all the time. It is quite common to see PhDs with experience in several different domains. For example, there are chemistry PhDs who have moved into computer science and mathematics PhDs who have become ecologists. Most research is inter-disciplinary anyway.
I hope I have provided you with some answers.