I believe I have come up with a way of bringing out detail using a similar technique. My variation on the LLRGB technique described above is much simpler to accomplish than the above procedure. My goal was not necessarily to bring out nebulocity that is weak, but instead to enhance the brighter areas of the nebulocity, to bring out detail which has faded when using the curves/levels functions in Photoshop. My method is good for emission nebula, which has most of its detail in the red channel.
I have used an image of the Eagle Nebula (M16) to demonstrate this simple procedure.
Here is the image, processed using the conventional means - Levels & Curves, and some of Neil Carboni's Astro tools.
This is the image, reporcessed from the same subs as the first image. There is more nebulocity visible, however, the goal in the processing was to enhance the detail in the core. I concentrated on that more than bringing out the fainter nebulocity. No doubt that if I had tried, I probably could have brought out more of the fainter stuff, but that was not the goal in this exercise.
What I did in this image was to take the original RGB image, stretch the curves and levels, but not too extreme. When the nebula is nicely visible in the image, I switched to channels, selected the RED channel only, selected and copied the red channel to a new image. This image is a monochrome (black and white) image of the red channel.
I enhanced this copy more with levels and curves, did some contrast curves, reduced star size and used an unsharp mask on the overall image. Your processing technique might differ, as I'm not an expert at image processing. The idea is to enhance the available detail. You want to enhance the grayscale version to show as much detail as possible.
On the original RGB version which should still be open in another window, blur it slightly, and saturate the colors. There are a few ways to do this, especially on an emission nebula, one of my favorite ways is to use curves on the red channel to enhance everything above the spike, while watching the total color histogram to make sure that you do not shift the overall color balance. Tone down the red in the stars by rolling off some of the red at the right end of the curve. It takes a little time with the curves, but you can get nice results. You can also use lab color, just make certain if you use that method, that you go into the exposure and increase the offset before you do the Lab color thing. Otherwise you can clip the red channel and ruin the image.
Once you have the color image saturated well, you can then go back to your monochrome image, select all of it, copy it and go back to the color image and paste the monochrome over the color image. It will paste the monochrome as a layer. Change the layer to a luminance layer. I found that in my image, I just left the opacity to 100%. If you need to back it down some, thats fine. When youre happy, flatten the image, do some small touch up work on the finished image, and youre done.