Symbolic links have existed for ages in UNIX, but Windows have not had any built in support for symbolic links until Vista. Vista has a utility called mklink, which allows you to create symbolic links for files and directories. Just be mindful that it requires admin privileges to run.
Symbolic links is useful for creating aliases for files or directories. Say you do a daily build of something. And you want to create an alias directory called current to point to the latest build you did. For simplicity, let's assume your build version is stored in some file called buildver.txt. You increment that every time you do a build. You create a directory named after the build version, and you create a current directory pointing to that.
if not exist buildver.txt echo 0 > buildver.txt
set /P CURRBUILD=<buildver.txt
set /A CURRBUILD=%CURRBUILD% + 1
REM the following assumes you have build.bat in your path that will create your directory content
mklink /D current %CURRBUILD%
This way a script that wants to access the current build can just always hit current without trying to figure out what the latest build version is.
If you want to delete it, you can safely use the usual rd command to remove directory. This will only delete the link, not the actual directory it points to. The same goes for file symbolic links. To remove it you use the del command and it will delete the link, not the file.
You can find out if a file is a link or not when you do a dir in a directory by the <SYMLINK> or <SYMLINKD> type it shows you.
For XP and Win2003, you can use a Sysinternals utility called Junction to create NTFS reparse point that will do similar job as a symbolic link. However unlike symbolic links which will work on files too, junctions only work for directories.
Another tool that can create junctions is the linkd utility from Win2k3 Resource Kit Tool which you can download.