Looking for a job in the open market is a paradox. One the one hand, you can see right in front of your eyes that there's a job available in a newspaper ad, on a sign in a window, on an Internet job board, or through a professional website. There's a job! There it is! I'll apply!
But on the other hand, there may not be a job. The job is advertised, yes, but in reality there's a "strong internal candidate" who has the job sewn up. Your solid-gold application, wherein you match to perfection your qualifications to the job's requirements, is not even given a chance, because the outcome of the search is known in advance. Schools, universities, government agencies and non-profits are notorious for advertising jobs that are essentially already filled; the advertisement is simply to meet a legal requirement.
Another way in which there any not be a job is that, indeed, there is no current opening. Businesses with high turnover (language schools, restaurants, any sales job that pays on commission) are constantly looking for new employees. They routinely publish ads for jobs that aren't available at the moment, but probably will be open soon.
The third way that there is no job is that, indeed, there's no job at all. Businesses sometimes post ads for jobs in their industry to get resumes; these resumes give them an idea of the competition's qualifications. And some very sneaky businesses post attractive ads anonymously, trying to catch their own employees looking for another job!
Of course, people get jobs every day through the open market, so there's nothing wrong with using it to job-hunt. Just don't be too disappointed if you don't get called in for an interview right away. It's no reflection on your qualifications--it's just the reality of the open job market.