Wednesday, December 22, 2010

Controlling Crawling and Indexing

Search engines generally have two main stages to make content available for users in search results. These are crawling and indexing. Crawling is the act of search engine crawlers accessing publicly available web pages. This involves looking at the web pages and following the links on those pages, just as a human user would do. Indexing involves gathering together information about a page so that it can be made available (“served”) through search results.
Automated website crawlers are powerful tools to help crawl and index content on the web. As a web master, you may wish to guide them towards your useful content and away from irrelevant content. The robots.txt file controls crawling, and the robots meta tag and X-Robots-Tag HTTP header element controls indexing. The robots.txt standard predates Google and is the accepted method of controlling crawling of a website.
  • How to use Robot.txt: It’s a simple text file to tell search robots which pages you don’t want them to index in search engine.
  • How to use Robots meta tags: It’s for those users who can’t control Robot.txt file like blogspot users. Blogger users can keep their content out of the search engine by using these robot meta tags.
  • How to use X-Robots-Tag Header: Just another method to restrict the access control of search engine. You can also prevent pdf file to index with this method.
  • Controlling Crawling

    The advantage with robots.txt file is it allows the specific path that you would like your site to be crawled. Crawlers request the robots.txt file from the server before crawling. Within the robots.txt file, you can include sections for specific (or all) crawlers with instructions (“directives”) that let them know which parts can or cannot be crawled. Location of the robots.txt file The robots.txt file must be located at the root of the website host that it should be valid for. For instance, in order to control crawling on all URLs below http://www.example.com/, the robots.txt file must be located at http://www.example.com/robots.txt. A robots.txt file can be placed on sub domains (like http://website.example.com/robots.txt) or on non-standard ports (http://example.com:8181/robots.txt), but it cannot be placed in a subdirectory (http://example.com/pages/robots.txt). More about Crawling and Indexing at Kensium Business Process Outsourcing

Knowledge Process Outsourcing

Knowledge Process Outsourcing (KPO), frequently compared to Business Process Outsourcing (BPO) is the next step in the outsourcing value chain. It is not an offshoot of BPO as people like to think but an independent industry. KPO is a form of outsourcing, in which knowledge-related and information-related work is carried out by workers in a different company or by a subsidiary of the same organization, which may be in the same country or in an offshore location to save cost. While BPO stresses on process expertise, KPO emphasizes business expertise. KPO requires advanced analytical and technical skills coupled with judgment to execute highly complex and customized processes.

Knowledge process can further be high added value processes chain where the achievement of objectives is highly dependent on the skills, domain knowledge and experience of the people carrying out the activity. And when this activity gets outsourced a new business activity emerges, which is generally known as Knowledge Process Outsourcing.

KPO involves a specialized domain knowledge of a higher level. It is a combination of BPO, Research Process Outsourcing (RPO) and Analysis Proves Outsourcing (APO). KPO Industry handles more amount of high skilled work other than the BPO Industry. While KPO derives its strength from the depth of knowledge, experience and judgment factor, BPO (Business Process Outsourcing) in contrast is more about size, volume and efficiency.

The main driver for outsourcing is cost reduction coupled with the value adds of these services. Labor cost savings overseas are just too great to be ignored. In knowledge-intensive industries such as Analytics and Data Mining Services, Research and Development, and Intellectual Property Research, companies can save significantly 40 to 50% by off shoring work to low-wage countries. On the value front, KPOs typically offer competitive advantage to its clients by providing market and industry research, data and statistical analysis, competitive analysis, and support in legal and administration processes.

Having one or more offshore centers, provides flexibility in terms of human resource and time management. Outsourcing allows companies to add or reduce personnel easily avoiding an expensive layoff process. By utilizing the time difference between different parts of the globe, development can take place constantly. The ability to send massive data amounts anywhere via the Internet allows a continual collaboration on a large scale.

There are some minor risks associated with KPOs which are well handled by KPO (Knowledge Process Outsourcing) organizations to ensure optimum benefits to the outsourcing companies. There is a tight wall of security in place during the data transfer and during the knowledge transfer. The confidentiality of the data is maintained by the KPOs that take due care to maintain high quality of work and personnel. A full fledged data back up is in place at the KPOs to make sure the outsourcing company can access whatever information they require at any time. Time lines are worked out in advance to accommodate the necessary smooth work flow.

There is a high potential for KPOs in the present day scenario. The scope of KPOs is not restricted to Information technology (IT) and Information Technology Enabled Services (ITES). KPOs have a future in other areas like Legal Processes, Intellectual Property and Patent Related Services etc. While the company concentrates on its core specialization and competency activities, it can outsource other activities and gain a lot of momentum in the core areas. There is a cost minimization this way and a total optimization in the business.