You need to make sure first that your Mac and the router are on the same IP network, since telnet is a TCP/IP protocol (for what it's worth, telnet typically operates on TCP port 23). Your router's documentation should say what its default IP address is. If you set up for DHCP and get an address other than 169.254.x.x you should be ok.
The command in terminal is simply "telnet". Your command line should look something like:
[athena:~] jason% telnet 192.168.1.1
then press return to send the command to the shell, which will invoke the telnet command and pass the single argument (the IP address) to it. telnet is engineered to interperet that argument to be the host you want to connect to, so it tries a connection. if the connection is successful, and the host at that IP address is listening for telnet connections, you'll be prompted to authenticate.
Once you authenticate, you'll be presented with the router's text-based user interface. it may be rudimentary menus that let you explore the options using arrows, enter, and escape. Or it may be just another prompt expecting you to know commands to use. In the latter case, it's probably best to hit the manufacturer's web site for a PDF explaining how to use the router's command line interface; they vary and there's no one consistent syntax. If you're lost, try typing help or ? at the prompt. The basic structure of commands will probably be the same: you enter the name of a command and the router responds appopriately; commands may require arguments to give the router something to work with.
Once you're done, use the router's exit command (or equivalent) to end the session gracefully. This will normally also end your telnet session too and give you your Mac OS X terminal prompt. Noting how your prompt changes as you use interactive programs like telnet will help you keep track of where you are as you use the terminal.
If you don't get a response when trying to connect, or need to get out of telnet because it keeps prompting for a password you don't know, you can make telnet give up the connection attempt by typing ctrl-]. Hold down the control key and press ]. The prompt then shows that you're still in the telnet program. To get back to your shell, type quit. If you're curious what other commands you can type at that telnet prompt, type help instead:
Commands may be abbreviated. Commands are:
close close current connection
logout forcibly logout remote user and close the connection
display display operating parameters
mode try to enter line or character mode ('mode ?' for more)
Hope this helps. -- Jason