What should a developer know before building a public web site?

The idea here is that most of us should already know most of what is on this list. But there just might be one or two items you haven't really looked into before, don't fully understand, or maybe never even heard of.

Interface and User Experience

• Be aware that browsers implement standards inconsistently and make sure your site works reasonably well across all major browsers. At a minimum test against a recent Gecko engine (Firefox), a Webkit engine (Safari, Chrome, and some mobile browsers), your supported IE browsers (take advantage of the Application Compatibility VPC Images), and Opera. Also consider how browsers render your site in different operating systems.
• Consider how people might use the site other than from the major browsers: cell phones, screen readers and search engines, for example. — Some accessibility info: WAI and Section508, Mobile development: MobiForge
• Staging: How to deploy updates without affecting your users. Ed Lucas's answer has some comments on this.
• Don't display unfriendly errors directly to the user
• Don't put users' email addresses in plain text as they will get spammed to death
• Build well-considered limits into your site - This also belongs under Security.
• Learn how to do progressive enhancement
• Always redirect after a POST.
• Don't forget to take accessibility into account. It's always a good idea and in certain circumstances it's a legal requirement. WAI-ARIA is a good resource in this area.

Security

• It's a lot to digest but the OWASP development guide covers Web Site security from top to bottom
• Know about SQL injection and how to prevent it
• Never trust user input (cookies are user input too!)
• Encrypt Hash and salt passwords rather than storing them plain-text.
• Don't try to come up with your own fancy authentication system: it's such an easy thing to get wrong in subtle and untestable ways and you wouldn't even know it until after you're hacked.
• Know the rules for processing credit cards. (See this question as well)
• Use SSL/HTTPS for login and any pages where sensitive data is entered (like credit card info)
• How to resist session hijacking
• Avoid cross site scripting (XSS)
• Avoid cross site request forgeries (XSRF)
• Keep your system(s) up to date with the latest patches
• Make sure your database connection information is secured.
• Keep yourself informed about the latest attack techniques and vulnerabilities affecting your platform.
• Read The Google Browser Security Handbook
• Read The Web Application Hackers Handbook

Performance

• Implement caching if necessary, understand and use HTTP caching properly as well as HTML5 Manifest
• Optimize images - don't use a 20 KB image for a repeating background
• Learn how to gzip/deflate content (deflate is better)
• Combine/concatenate multiple stylesheets or multiple script files to reduce number of browser connections and improve gzip ability to compress duplications between files
• Take a look at the Yahoo Exceptional Performance site, lots of great guidelines including improving front-end performance and their YSlow tool. Google page speed is another tool for performance profiling. Both require Firebug installed.
• Use CSS Image Sprites for small related images like toolbars (see the "minimize http requests" point)
• Busy web sites should consider splitting components across domains. Specifically...
• Static content (ie, images, CSS, JavaScript, and generally content that doesn't need access to cookies) should go in a separate domain that does not use cookies, because all cookies for a domain and it's subdomains are sent with every request to the domain and its subdomains. One good option here is to use a Content Delivery Network (CDN).
• Minimize the total number of HTTP requests required for a browser to render the page.
• Utilize Google Closure Compiler for JavaScript and other minification tools
• Make sure there’s a favicon.ico file in the root of the site, i.e. /favicon.ico. Browsers will automatically request it, even if the icon isn’t mentioned in the HTML at all. If you don’t have a /favicon.ico, this will result in a lot of 404s, draining your server’s bandwidth.

SEO (Search Engine Optimization)

• Use "search engine friendly" URL's, i.e. use example.com/pages/45-article-title instead of example.com/index.php?page=45
• Don't use links that say "click here". You're wasting an SEO opportunity and it makes things harder for people with screen readers.
• Have an XML sitemap, preferably in the default location /sitemap.xml.
• Use when you have multiple URLs that point to the same content
• Use Google Webmaster Tools and Yahoo Site Explorer
• Install Google Analytics right at the start (or an open source analysis tool like Piwik)
• Know how robots.txt and search engine spiders work
• Redirect requests (using 301 Moved Permanently) asking for www.example.com to example.com (or the other way round) to prevent splitting the google ranking between both sites
• Know that there can be bad behaving spiders out there
• If you have non-text content look into Google's sitemap extensions for video, etc. There is some good information about this in Tim Farley's answer.

Technology

• Understand HTTP and things like GET, POST, sessions, cookies, and what it means to be "stateless".
• Write your XHTML/HTML and CSS according to the W3C specifications and make sure they validate. The goal here is to avoid browser quirks modes and as a bonus make it much easier to work with non-standard browsers like screen readers and mobile devices.
• Understand how JavaScript is processed in the browser.
• Understand how JavaScript, style sheets, and other resources used by your page are loaded and consider their impact on perceived performance. It may be appropriate in some cases to move scripts to the bottom of your pages.
• Understand how the JavaScript sandbox works, especially if you intend to use iframes.
• Be aware that JavaScript can and will be disabled, and that Ajax is therefore an extension not a baseline. Even if most normal users leave it on now, remember that NoScript is becoming more popular, mobile devices may not work as expected, and Google won't run most of your JavaScript when indexing the site.
• Learn the difference between 301 and 302 redirects (this is also an SEO issue).
• Learn as much as you possibly can about your deployment platform
• Consider using a Reset Style Sheet
• Consider JavaScript frameworks (such as jQuery, MooTools, or Prototype), which will hide a lot of the browser differences when using JavaScript for DOM manipulation

Bug fixing

• Understand you'll spend 20% of the time coding and 80% of it maintaining, so code accordingly
• Set up a good error reporting solution
• Have some system for people to contact you with suggestions and criticism.
• Document how the application works for future support staff and people performing maintenance
• Make frequent backups! (And make sure those backups are functional) Ed Lucas's answer has some advice. Have a Restore strategy, not just a Backup strategy.
• Use a version control system to store your files, such as Subversion or Git
• Don't forget to do your Unit Testing. Frameworks like Selenium can help.

Lots of stuff omitted not necessarily because they're not useful answers, but because they're either too detailed, out of scope, or go a bit too far for someone looking to get an overview of the things they should know. If you're one of those people you can read the rest of the answers to get more detailed information about the things mentioned in this list. If I get the time I'll add links to the various answers that contain the things mentioned in this list if the answers go into detail about these things. Please feel free to edit this as well, I probably missed some stuff or made some mistakes.

link|flag edited Nov 23 at 16:20
community wiki
73 revs, 33 users 31%
Joel Coehoorn
34   Then edit it. I didn't write most of this: I'm only maintaining it -- a job which I've inherited because I asked the question, solicited this larger answer specifically, and I'm genuinely interested in seeing what we can come up with. The more contributions the better. – Joel Coehoorn Mar 16 '09 at 1:18 92   One more note: if you do come back and edit this, try to be respectful of what was written. Don't just remove the parts you disagree with: actually take the time to address the short-comings and provide something better. – Joel Coehoorn Mar 16 '09 at 1:19 9   This is awesome, thanks to everyone involved in editing it. – Fire Crow Mar 24 '09 at 15:07 4   To the SEO/"don't use 'click here' links" topic: use "'log in' to post comments" form instead. – erenon Jun 13 '09 at 9:57 15   @Carlo: If you really want to be a web developer then yes you really should as an individual understand everything in this list. – Chris Lively Feb 9 at 22:23 show 16 more comments up vote 209 down vote

Rule number one of security:

Never trust user input.

link|flag answered Sep 16 '08 at 16:03
community wiki
Jonny Barnes
87   Added to which: Cookies count as user input. – Colonel Sponsz Nov 20 '08 at 14:19 25   +1 users are evil – Yassir May 8 '09 at 17:56 9   Assume users are idiots. If the user gives you input as expected, treat that as the exception and not the rule. A corollary: don't expect users to read instructions either, no matter how short, simple, and obvious they are. – bta Mar 15 at 20:11 9   HTTP headers count as user input too (including the referer). – Tgr Jun 16 at 22:31 3   I thought the first rule of security was don't talk about security. – Moses Oct 5 at 20:11 show 4 more comments up vote 143 down vote

• Never put email addresses in plain text because they will get spammed to death
• Be ultra paranoid about security.
• Get it looking correct in Firefox first, then Internet Explorer.
• Avoid links that say "click here". It ruins a perfectly good SEO opportunity.
• Understand that you'll spend 20% of the time developing, 80% maintaining, so code it accordingly.

link|flag answered Sep 16 '08 at 14:15
community wiki
Aston
4   "Click here to blah" may be good if you expect many inexperienced users who may not realize where exactly they can click. – LKM Oct 13 '08 at 8:47 31   +1 for Fox -> IE : you don't make a car that works in the arctic and reverse engineer for city driving – annakata Nov 20 '08 at 14:24 11   I have my email address on my web site, and my spam filters have been perfectly adequate for the task, except for the backscatter spam which was generated from domains I owned. – David Thornley Dec 31 '08 at 18:22 4   @David Thornley: Hey, it's great that you have efficient ways to block spam for your email address, but the point was not to expose the users' email addresses. Do that and you'll have an angry mob at your doorstep faster than you could say 'Spam"! BTW, this sounds to me like basic privacy courtesy. – Cristi Diaconescu Nov 4 '09 at 18:32 7   Get it working in Chrome first. It's web kit and a good chance you are doubling up on the Mac users of your site. Then FF then IE. – Marc Feb 9 at 22:21 show 10 more comments up vote 87 down vote

• Web standards: it's cheaper to aim for standards than testing for every browser available (and in a public website you will see a lot of different browsers/version/OS combinations (30+))
• SEO-friendly URLs: changing URLs later in the game is quite painful for the developers and the site will most probably take a PageRank hit.
• Understand HTTP. If you have only worked with ASP.NET webforms, then you probably don't really understand HTTP. I know people that have worked with webforms for years and they don't know the difference between a GET and a POST, let alone cookies or how session works.
• HTTP Caching: Understand what to cache and what NOT to cache.
• Optimize image weights. It's not cool to have a 20 KB image for a repeating background...
• Read and understand Yahoo's best practices (http://developer.yahoo.com/performance/rules.html). Not every rule applies to every website.
• Use YSlow for guidance, but understand its limitations.
• Understand how JavaScript is processed on the browser. If you put tons of external scripts at the beginning of your page, it's going to take forever to load.
• Consider cell phone usability: some users will access your site using their native cell phone browser (I'm not talking about iPhones or Opera Mini). If your site is pure Ajax, they will probably be out of luck.
• Learn the difference between 301 and 302 redirects: it's not the same for search engines.
• Set up Google Analytics (or any other analytics package) right from the start.

Not specific to public websites, but useful nevertheless:

• Server caching: identify and exploit any caching opportunities, it makes a big performance difference. It's often overlooked on non-public websites.
• Set up a good error reporting solution, with as many details as possible. You will get a lot of errors when you launch, no matter how much you tested, so you better get all the details you can.
• Set up an Operation Database (see for example http://ayende.com/Blog/archive/2008/05/13/DevTeach-Home-Grown-Production-System-Monitoring-and-Reports.aspx) so you can quickly identify bottlenecks.
• Set up a good deployment strategy. You will probably deploy more often than non-public sites (we deploy daily).
• Realize that web applications are inherently multi-threaded, you will have lots of visitors (typically much more than in non-public websites), and threads are not unlimited.

link|flag edited Feb 9 at 22:12
community wiki
2 revs, 2 users 73%
Mauricio Scheffer


show 2 more comments up vote 75 down vote

Security

• Filter and validate incoming user input ('amount' does not need to accept alphabetical characters) and escape outgoing user input (a ' in user input, is NOT the same as an SQL ').
  Never trust any data given by the user.
• And the above will help with protecting against SQL injection.
• Understand SSL
• Keep your systems up to date with the latest patches.
• Protect yourself from cross site scripting
• How to resist session hijacking
• Find out about HTTPOnly cookies
• How to handle authentication/permissions
• Understand PKI (public keys)
• Keep up to date! This is the most important thing, make sure to follow all the latest information about possible security issues and vulnerabilities that affect your platform.

SEO

• Create SEO friendly URLs - example.com/articles/rampaging-bull-tramples-unicorn NOT example.com?article=45
• Use an XML sitemap so that site engines can crawl your site more intelligently
• Set up Google Analytics (or another analytics package) from the start
• Learn the difference between 301 and 302 redirects: it's not the same for search engines.
• Set up a robots.txt file

Performance

• How to cache
• What not to cache
• How to gzip
• Make regular backups. Don't just rely on your hosting provider - have another backup source in case something critical is destroyed (like a database table)
• Read Yahoo's best practices (http://developer.yahoo.com/performance/rules.html) for information on improving performance
• Set up an Operation Database (http://ayende.com/Blog/archive/2008/05/13/DevTeach-Home-Grown-Production-System-Monitoring-and-Reports.aspx) to quickly identify bottlenecks.
• Look into performance monitoring

Productivity

• Documentation!
• Code from the beginning with maintainability in mind
• Have a good deployment strategy - don't save it to the very end to figure this out.
• URLs designed with REST in mind could save you a headache in the future.
• Use patterns like MVC to separate your application flow from your database logic.
• Be aware of the many frameworks out there that will speed up your development
• Use staging and a version control system to deploy updates so that your users won't be affected
• Set up an error logging system. No matter how well coded your website will have errors when it is released. Don't wait for the user to let you know; be proactive in identifying errors and bugs
• Have a bug tracker
• Know your environment. Your OS, language, database. When you need to debug it will be important to understand how these things work at a basic level in the least.

User experience

• Be aware of accessibility. This is a legal requirement for some programmers in some jurisdictions. Even if it's not, you should bear it in mind.
• Never put email addresses in plain text, or they will be spammed to death.
• Have some method for users to submit their comments and suggestions
• Catch errors and don't display them to the user; display something they can understand instead
• Remember that cell phones and other mobile devices with browsers are becoming more common. Sometimes they have very poor JavaScript support. Will your site look okay on one of these?

Core Web technologies

• Understand HTTP, and things like GET, POST, cookies and sessions.
• How to work with absolute and relative paths
• Realize that web applications are inherently multi-threaded, you will have lots of visitors (typically much more than in non-public websites), and threads are not unlimited.