The JS file is too large? Feed it to your favorite JS minification tool, such as Google Closure Compiler.
Slow display output? If you use a rectangular layout, you can enable glyph caching by calling ROT.Display.Rect.cache = true. This enables rendering into tiny cacheable canvases, but will improve performance only if you do not use too many glyph/color/bgcolor combinations. Your mileage may vary.
Too much time spent in lighting computations? Decrease the maximum light range or switch to 1-pass lighting.