Blogdex adds gzip support
Downtime
On Blogdex exploitability
Announcing Tracking Syndication
Domain name change
MySql error
Infrastructure changes
Taking the garbage out
RSS and Uniqueness
May 2004
December 2003
August 2003
July 2003
June 2003
April 2003
March 2003
January 2003
December 2002
November 2002
October 2002
August 2002
June 2002
May 2002
April 2002
March 2002
February 2002
January 2002
December 2001
November 2001
October 2001
September 2001
August 2001
Welcome to the new Blogdex!
So after quite a few late nights I've decided to put up the new version of Blogdex. I hate to release partially complete and buggy code, but I've been lazy about doing the frontend and this is a way to force me to finish things. If you notice any quirky behavior, or have suggestions, please post them here.
I'll do my best to outline all of the changes and things to come:
Done
Back-end:
I spent quite a bit of time over the past few weeks designing and iterating on the back-end. Under the pressure of scaling issues, I've resolved pretty much all of the outstanding storage and memory constraints so that all of the old functionality will come back. As far as I can tell, the back end is elegant and efficient, and shouldn't need much work for the next 10 years :)
Complete database redesign, reducing overall data size from 1.5GB to just around 300MB
New data: history for all previous top 50 pages, an index of all blog-related sites, and an index of social weather for the past year
A data agent architecture, allowing for simultaneous crawling, parsing, and representation updates, which also makes adding new structure quick and easy. As of right now there are 8 agents including a weblogs.com update crawler, link parser, a few representational updaters, and a title crawler. This has been more or less in place for a few weeks, and it's running very smoothly. Right now Blogdex trails Weblogs.com in recency by about 5 minutes.
Front-end:
New design (ducking for cover)
Templated HTML construction, which makes my life much easier
A new terser top 50 page, with more information hidden in the mouseover of the link
To do
I should take care of a big chunk of this tonight, some of this is mission-critical. These are in relative order of importance:
1. Weblog add page and the RSS 0.91/1.0 pages aren't fully function. Eek!
2. Search. I was kind of wary of putting this up at all without any search capabilities. I know from my own interests that people won't be happy for long not being able to find themselves in the database.
3. History. The "a year ago today" is a small glimpse into something I am really excited about. I now have archives of every time a site was in the top 50, which gives quite a bit of context to the project.
4. Charts. Thanks to a generous gift from the people at Chart Director, I have a nice, simple graphics package for charting link behavior over time.
5. Top blogs: now that I have an index of all pages related to a given weblog, it's much easier to generate the popularity contest to end all popularity contests. I'm sure this would be priority one if the users were in control :)
6. Social weather: the data exists now to give a social weather "forecast" based on various statistics around weblog linking behavior (how many links, how convergent, how many are to blog sites, etc.).
I'm also not content with a lot of the design of the front end, so expect some major tweaking there. Please let me know if you have any suggestions.
Now I'm going to sleep. G'night!
Comments
Great job Cameron, Congrats!
I remember that when a couple of Persian posts appeared on top of the list a few months ago, we realized that blogdex is only an English-friendly tool.
I remember a brief discussion at the time that what if you could present blogdex in different interfaces based on the user's desired language.
How about doing it for the new version? We can easily use HTML language-related meta tags to separate them.
A choice of font sizes on the main page would be lovely, and probably easy to implement. The text for the links and descriptions is just a tad too large in the default template. Other than that, a grand job. Well done...
The dashed line under links makes it difficult to scan. Also, you could decrease the size of the link name, and add space between links to avoid one big clump. Otherwise, awesome!
Marvelous. It looks nice, is faster and with some new functionality on the front page. Could we ask for more:-)
If you need a mirror I've got quite a bit of spare room at my server. I know you're at the Media Lab so it shouldn't be an issue but anyways, if you need it let me know.
Rikard
Oh and if I were you I'd just rip the text-sizing functionality off wired.com
I can't figure out how to see more than the top 50 entries.
unfortunately, our blog service provider, persianblog.com, has got some technical problems these days, ( actually i dont know whats the real problem )
i am sure if this didnt happan the most hitted blog in your list would be kalagh@persianblog.com, because of dying this dude,
i think the most organized group og bloggers in the world are iranian, hereby i wanna thank Mr Heseyn Derakhshan, ( as the bginner ) and persianblog.com, ( as the first iranian blog server )
regards
B H Yousefi
There's something odd with your stylesheet. Looking at this page: http://blogdex.media.mit.edu/news/archives/2002_10.html#000106 the text disappears off the left of my screen by a few characters. I'm using Mozilla 1.2b on Windows XP.
Looks purty nice! I like the black text on white background.
On the constructive criticism side, I second the comment on allowing the page to fold up nice at 800 by 600--some us still use old laptops! If you can do it, I also suggest abandoning layout tables and presentational markup in favor of pure CSS--and using a real ordered list for records.
Also, how does one page to records below the top 50? Often I don't really bother to read the top 50 because it's stuff that I've already heard about in on form or another. Blogdex is really cool when you start browsing 8 or 9 pages in.
If you set the table width to 780px instead of 800px, it'll look just perfect in 800x600 resolution. Just a tip. Good work otherwise!
Ah refreshing..... time to get down to browsing the new bd2 ;) thxs for the effort cam.
Please check out the sites about poker poker http://www.middlecay.org/ online poker online poker http://www.hasslerenterprises.org/ phentermine phentermine http://www.teambeck.org/ viagra viagra http://www.neweighweb.org/ casino casino http://www.hdic.org/ online casino online casino http://www.hometeaminspection.org/ texas holdem texas holdem http://www.mor-lite.org/ texas hold em texas hold em http://www.reservedining.org/ pacific poker pacific poker http://www.paramountseedfarms.org/ empire poker empire poker http://www.parkviewsoccer.org/ party poker party poker http://www.tclighting.org/ cialis cialis http://www.sportingcolors.org/ diet pills diet pills http://www.psychexams.org/ tramadol tramadol http://www.stories-on-cd.org/ play poker play poker http://www.marshallyachts.org/ turbo tax turbo tax http://www.lvcpa.org/ pacific poker pacific poker http://www.rifp.org/ texas hold em poker texas hold em poker http://www.suttonjames.org/ poker rules poker rules http://www.pagetwo.org/ credit cards credit cards http://www.tecrep-inc.org/ hoodia hoodia http://www.rethyassociates.org/ carisoprodol carisoprodol http://www.ingyensms.org/ payday loans payday loans http://www.krantas.org/ buy phentermine buy phentermine http://www.devilofnights.org/ generic viagra generic viagra http://www.ansar-u-deen.org/ h r block h r block http://www.azian.org/ buy viagra buy viagra http://www.twinky.org/ texas holdem poker texas holdem poker http://www.atlanta2000.org/ weight loss diet pills weight loss diet pills http://www.catchathief.org/ weight loss weight loss http://www.nehrucollege.org/ casino casino http://www.casinoequipmentsalesandrental.com/ ... Thanks!!!
You are invited to check the sites in the field of online poker online poker http://www.mcdortaklar.com/ phentermine phentermine http://www.reservedining.net/ viagra viagra http://www.paramountseedfarms.net/ credit cards credit cards http://www.rethyassociates.net/ casino casino http://www.ingyensms.net/ poker poker http://www.bigyonet.com/ online casino online casino http://www.zalaszentgrot.com/ texas holdem texas holdem http://www.darkangelclan.com/ texas hold em texas hold em http://www.middlecay.net/ texas holdem poker texas holdem poker http://www.hasslerenterprises.net/ pacific poker pacific poker http://www.hdic.net/ party poker party poker http://www.hometeaminspection.net/ empire poker empire poker http://www.mor-lite.net/ poker games poker games http://www.parkviewsoccer.net/ generic viagra generic viagra http://www.targetindustries.net/ cialis cialis http://www.tclighting.net/ levitra levitra http://www.neweighweb.net/ tramadol tramadol http://www.jfcadvocacy.net/ online pharmacy online pharmacy http://www.psychexams.net/ soma soma http://www.stories-on-cd.net/ diet pills diet pills http://www.lvcpa.net/ phendimetrazine phendimetrazine http://www.suttonjames.net/ credit card credit card http://www.mp-forum.com/ payday loans payday loans http://www.devilofnights.net/ loans loans http://www.gargzdai.net/ personal loans personal loans http://www.zone-b51.com/ student loans student loans http://www.jmsimonr.com/ private mortgages private mortgages http://www.1a1merchantaccounts.com/ low interest credit cards low interest credit cards http://www.at-capstone.com/ ... Thanks!!!
Hello folks nice blog youre running
What is freedom of expression? Without the freedom to offend, it ceases to exist.
How about narrowing the content a little? I'll leave alone the fact that you're using a table for layout, when it's really unnecessary.... But 800px is a bit wide. How about using percentages instead? Let the content adjust itself to the user's preferred browser width.