Content Discovery
This function can be used to discover content and functionality which is
not linked from visible content that you can browse to or spider.
To access this function, select an HTTP request anywhere within Burp, or
any part of the Target
site map, and choose "Discover content"
within "Engagement tools" in the context menu.
Burp uses various techniques to discover content, including name
guessing, web spidering, and extrapolation from naming conventions observed
in use within the application. Discovered content is displayed within a
special site map that is specific to the discovery
session, and can also optionally be added to the main
suite site map.
Control
This tab shows you the current status of the discovery session.
The toggle button indicates whether the session is running, and lets you
pause and restart the session.
The following information is displayed about the progress of the
discovery session:
- Number of requests made
- Number of bytes transferred in server responses
- Number of network errors
- Number of discovery tasks queued
- Number of spider requests queued
- Number of responses queued for analysis
The individual discovery tasks that are queued are shown in a table. The
discovery engine works recursively, and when a new directory or file is
discovered, further tasks are derived from this, depending on the
configuration. For example, when a new directory is discovered, Burp might
add tasks to look for sub-directories and files within that directory; or,
when a new file is discovered, Burp might add a task to check for the same
base filename with different file extensions. Newly added tasks are
prioritized according to their likelihood of quickly discovering new
content.
Target
These options let you define the start directory for the content
discovery session, and whether files or directories should be targeted. The
following options are available:
- Start directory - This is the location where Burp
will start looking for content. Only items within this path and its
subdirectories will be requested during the session.
- Discover - This option determines whether the
session will look for files or directories or both. If you are checking
for directories, you can choose whether and how deep to recurse into
discovered subdirectories.
Filenames
These options let you configure the sources that Burp should use for
generating filenames to test. The following options are available
- Built-in short file list
- Built-in short directory list
- Built-in long file list
- Built-in long directory list
- Names discovered in use on the target site. If this option is
selected, Burp will maintain a list of all directories and filename
stems that have been discovered on the target site, and will also check
for these in each new directory that is tested.
- Derivations based on discovered items. If this option is selected,
Burp will attempt to guess item names based on those that have already
been discovered. For example, if the directory AnnualReport2011 is
discovered, Burp will also check for AnnualReport2012, AnnualReport2013,
etc.
File Extensions
These settings control how the discovery session adds file extensions to
file stems that are being tested. The file stems themselves are derived
according to the filenames options. When each file
stem is tested, Burp check for various different extensions, according to
these settings. The following options are available:
- Test these extensions - This option lets you
configure a list of extensions that Burp will always check for. You can
fine-tune the default list based on the technologies known to be in use
on the target application.
- Test all extensions observed on target site - If
this option is selected, then Burp will automatically check for file
extensions that have been observed in use on the target site. This
option is useful when you don't know exactly what extensions or
technologies are in use. You can also configure a list of extensions
that you don't want to check for even if found to be in use (such as
image files).
- Test these variant extensions on discovered files -
This option lets you configure a list of extensions that Burp will
additionally check for using the stems of discovered filenames. This
option is useful to check for backup copies of existing files.
- Test file stems with no extension - If this option
is selected, Burp will check for each file stem with no extension added.
Discovery Engine
These settings control the engine used for making HTTP requests when
discovering content, and interaction with the suite
site map. The following options are
available:
- Case sensitivity - This setting controls whether
Burp will handle filenames case sensitively. If "Auto-detect" is
selected, then Burp will start by handling filenames case sensitively,
and on discovering the first new item, will test the server's treatment
of case variations. Depending on that treatment, Burp may revert to
handling filenames case insensitively.
- Add discovered content to suite site map - If this
option is selected, then new items identified in the current
discovery session will be automatically added to the main
suite site map.
- Copy content from suite site map - If this option
is selected, then the discovery session will copy any existing relevant
content from the main suite site map
into the discovery site map, to provide a
stronger starting basis for discovering new content.
- Spider from discovered content - If this option is
selected, then the discovery session will perform conventional web
spidering, and will process the responses to discovery requests looking
for links to additional new content.
- Number of discovery threads - This option controls
the number of concurrent requests the discovery engine is able to make.
- Number of spider threads - This option controls the
number of concurrent requests the spidering function is able to make, if
enabled.
Site Map
The discovery session employs its own site map, showing all of the content
which has been discovered within the defined scope. If you have configured Burp
to do so, newly discovered items will also be added to Burp's main
site
map.