Q. Does the scripting language have to be Javascript?
A. Yes. There are other somewhat supported scripting languages, but JavaScript is by far the easiest to get into, most widely supported, and most widely documented. In terms of JS toolkits available, jQuery is certainly an option. So is Dojo Toolkit; with its AMD design scheme, it's (arguably) the better choice if you're looking to develop long-term. It's the toolkit I decided to go to when developing my own game engine, TGE.
Q: Are there any great guides you guys know of?
A: Yes! The Mozilla Developer Network is a great guide on all things HTML5. It will provide you with the basic skills necessary to manipulate HTML5 DOM elements. However, game development theory is still king; you'll at least need to understand things like separating the physics and animation loops, sprite animation, scene management (yes, PONG has a scene, and there are objects in it which need management). Then there are paradigms like event-driven (vs. function-driven) programming, callbacks, the execution context stack, and so on. I'd suggest reading through The Good Parts, The Definitive Guide (if you've got time), and Patterns.
Q: What are some good conventions I should be aware of?
A: I chose the "do everything in JavaScript" path, and avoid authoring DOM elements in HTML where possible. I'm not sure if this is highly desirable, but it makes the most sense to me to keep as much in JS where possible. YMMV
Breaking down your game into Modules ("singleton"-esque interface-style, uh, interfaces like Graphics and Network and Audio) and Factories (which produce Objects like Sprites and Bullets and ParticleEmitters and PowerUps) is crucial for debugging, and important for organizing your code, especially as your project grows. A toolkit like Dojo (and to some extent, RequireJS) trumps jQuery in terms of usefulness in this regard, which is why I highly tout it.
Develop using Google Chrome, and use Chrome Developer Tools (CDT) to debug your application. Seriously. Firefox+Firebug is nice, but Chrome+CDT beats the pants off it.
Use requestAnimationFrame in your render loop.
Write your game such that it will function in all browsers which support the bare minimum elements you're planning on using. That does not mean supporting IE6, but it does mean making sure your game functions in IE9/10, Firefox, Chrome, and possibly Safari and Opera. You'll learn a lot on making your game compatible with different browsers. Don't waste your time on backward compatibility (or "progressive enhancement"), as people who are using old browser (aside from, say, your parents) won't even be looking at your game.
Trailing comma of death is your enemy.
If you're developing in Windows, Notepad++ is a great editor, and XAMPP is a great web-server stack.
If you're going to eventually write some server-side code, PHP, C#, are great options for server-side scripting languages. Other very popular languages include Ruby, but my experience is limited to the two former, so I won't comment more than that.
Q: What's the best way to get sound?
A: Support OGG and MP3, and you'll cover the bases you need to enable playback in the Big Three browsers. Create your audio tags once at application startup, keep the audio elements around a long time, and reuse them as often as you can, to avoid manipulating the DOM excessively.
Good luck, and happy JavaScripting!