List Books
Today I officially launch yet another new site. List Books is designed to allow anyone to sell their second-hand books online with no fees to list them and no commission when you sell them. I have been offering my books online for a while through my List Stuff site, which I built because I couldn’t find an online service that didn’t want a big chunk of money to allow me to sell books through their site. Great for me, I can build my own, but I had thought of making it available to anyone and I was spurred into action by some newly launched sites aimed specifically at university students wanting to sell their old textbooks. These sites were warmly received, and they’re a clever idea. I have no wish to compete with them, though there is nothing to stop people listing textbooks on my new site, but I was aiming at a different audience: those people, like me, who seem to have read quite a few books they don’t want anymore and would like to get rid of for one reason or another.
If you’re only interested in buying and selling second-hand books, head over to the site and either start browsing or sign up and start listing, if you’re interested in how I built the site, read on.
From Concept to Reality
I started out thinking it would be easy enough, all I’d need to do was add a user registration system to the code that runs List Stuff and away we go. Boy was I wrong. You see, list stuff had plenty of things, all the back end was working (I don’t usually bother building fully working admin systems for sites only I will use) for example, so you could add, edit and delete items, you could see what books had been sold and when and I even had a page that emailed me after taking payment via Nochex. I knew I had some other stuff to tidy up though. The listings page, for example, would need redesigning and some way to select books by author or subject would also be useful, but not much else. When I started thinking about it, however, I kept seeing more problems with the current system.
You see, the code and underlying database that ran List Stuff was designed to allow listings for one person, so when you scaled it up, it didn’t work too well. One of the problems was repetition. Say I had a hundred users, each with 10 books a piece. That’s 1000 books, or is it? You see, chances are, more than one person is going to be selling the same book. One of the overriding philosophies of database design is data normalisation. In essence, what it means is that you don’t repeat data within the database. Repeated data does more than bloat your database, it leads to data corruption. It also caused problems for people searching for items, they could be listed any number of ways. So that meant storing the book details separately and then linking all items for the same book to that information. Reading up on database normalisation I attempted to create the database correctly, I think I’m pretty much there.
To start with, I made a list of necessary features and some ‘nice to haves,’ and the list just seemed to grow. Most of them were usability based, things like making books easy to find in a number of different ways (genre, author, best rated, etc) and making it easier to add books (having added 50 of my own to the List Stuff site, I definitely wanted this). I realised that the first thing would be to get data into the system, so I started on the user/item admin side of the site first.
Normally, getting input from a web page into a database is fairly easy, you create a form, ask the user to fill in the details, check that they haven’t done something completely foolish to the data and then create a new record with that data. This was a little more complicated. Firstly, the book’s details needed to be put somewhere else, assuming they weren’t already in the system. Then you need to create the item – the listing for the physical book. Not so bad. Authors, however, proved a lot more fun. Most books are written by one person, which makes life easy. Thinking about it now I probably could have gotten away with just saving them with the book, but technically it was duplication, and it caused problems for searching or listing books by author. This meant some fancy footwork to break up and save them, however many there were, separately, along with a reference back to the book. Sounds easy, it wasn’t, for me at least, because of the permutations (i.e. names separated by a comma, or & or the word ‘and’). It also makes it more fun putting them back together again to display with a book or item.
Next, I went in search of a pre-built script that would resolve ISBN numbers into book details, figuring there would be plenty about. Wrong again. So I looked into writing my own, but there is no central database of ISBNs (to my surprise, someone issues them, they must have a complete list), so I looked into getting the data from various online bookstores, most notably, Amazon. That’s when I ran into Amazon’s web services (AWS), which is an excellent service for allowing developers to query Amazon.com’s complete catalogue using almost any type of search. It’s pretty simple to use and completely free, making it a great resource. Unfortunately, it’s listings are aimed at the US and it didn’t pick up many of the ISBNs for UK editions of popular books, to complicate matters, Amazon.co.uk doesn’t offer this service, so I then I had to find another way to do it. PHP scraping was the answer.
This basically entails reading a page with PHP as a text file, then extracting the relevant information using various string handling functions. I haven’t found any books, as yet, that these two don’t pick up, though I do have a couple of fall back sites should the going get tough (I just haven’t been able to test them yet). I had actually built the script as a standalone, so it needed to be integrated into the site to allow this functionality during the ‘add item’ process.
The rest of the admin section was relatively straight-forward and just a case of adapting the code I already had, although some other functionality was needed. I added the ability to track orders for example, which I have found useful on some of the sites I’ve used. This means that a user can track his or her orders or sales to see when they were sold, when they were dispatched and when they were received. Obviously it relies on the end users to make use of the system though.
I also wanted reviews and rating of the books on offer, having the book details stored separately makes this a lot easier, how could you review an item which might only be there a short time? You could, but you’d lose all the reviews the minute it was sold (well, you could get around this). Working out how to store the ratings and to increment them to make sure that the ratings were roughly right (i.e. so that if someone rates a book as one star but it has 20 people rating it at five stars it doesn’t immediately drop to three stars) was another challenge. User feedback, which I wasn’t going to include to start with, proved much easier to implement, thankfully, but as people don’t need to register for an account in order to buy things, it posed something of a security problem, with anyone able to write feedback if they knew what they were doing. I solved by using a check string and asking them to confirm their email address.
Lastly came form validation (sites would be a whole lot easier if you didn't have to rely on user input). Originally I was going to use Simon Willison's form validation class, but in the end I gave up and went the old fashion route. Form validation is not my favourite thing, I doubt it's any programmers favourite thing.
The site is far from perfect and I’ll need to continually tweak and refine it, not to mention adding some more features as time and circumstances allow. Overall I am pretty happy with the site, it certainly pushed my coding abilities further than I thought. Hopefully it’ll prove popular and, more importantly, useful.