Why I Hate Django
Cal Henderson delivered a keynote titled "Why I Hate Django" at the first annual DjangoCon. Django is an open source bsd licensed web framework written in python. Google has posted the keynote in its entirety to Youtube, which you can find embedded above. While the talk is humorous (and takes many jabs at rails developers) it does provide insight into what makes a good web framework. Cal is director of engineering at flickr and is an authority on how to make websites scale. He points out that most frameworks are designed to get projects off the ground quickly, but are lacking when it comes to building an even larger service. He talks about several things in django that need work and improvements that could be made. It’s really an interesting look at what it takes to go big.Towards the end, Cal notes that django has no mascot, “I could take a framework seriously that had a mascot with magical powers.” well, that seemed to be the easiest thing to fix so avalonstar created django: the web framework for ponies with magical powers. It’s available as wallpaper on djangopony, and ruby legend, _why, has already set about debunking the magic involved.Watch a video at http://www.bestechvideos.com/2009/04/08/djangocon-2008-keynote-cal-henderson
Content
Why I Hate Django
Cal Henderson
O HAI
DjangoCon 2008
2
My happy face
DjangoCon 2008 3
DjangoCon 2008
4
2.8 billion photos
DjangoCon 2008
5
2.8 billion photos
1 per 2.3 people on earth 9.2 per US resident
DjangoCon 2008 6
35k photos/sec >2m photos/min
DjangoCon 2008
7
Many thousand pages/sec
DjangoCon 2008 8
Scale nerd
DjangoCon 2008
9
Serious Language
DjangoCon 2008 10
Wordpress Yahoo
Mediawiki
DjangoCon 2008
Facebook
11
Wordpress Yahoo Digg Delicious
Mediawiki
DjangoCon 2008
Facebook
12
Flamework Serious Toolset
DjangoCon 2008
13
Flamework Flint Flinderbox Flaux Flag
DjangoCon 2008
Flink Flotos Flictionary Flops Fleng
14
Why hate Django?
DjangoCon 2008 15
What does he even know about Django?
DjangoCon 2008
16
Pownce
DjangoCon 2008 17
A fucking blogging engine
DjangoCon 2008
18
Why all the hate?
DjangoCon 2008
19
DjangoCon 2008
20
DjangoCon 2008
21
DjangoCon 2008
22
DjangoCon 2008
23
An attack on several fronts
DjangoCon 2008
24
An attack on several fronts
The team Python Frameworks
DjangoCon 2008 25
Django itself
Performance Scalability Maintainability
DjangoCon 2008 26
The Team
DjangoCon 2008 27
Frameworks
DjangoCon 2008
28
Struts
DjangoCon 2008 29
Craig McClanahan
DjangoCon 2008 30
DjangoCon 2008
31
Exhibit A : Beard
DjangoCon 2008
32
But Struts is for suits!
DjangoCon 2008 33
The Hot Shit
DjangoCon 2008 34
DjangoCon 2008
35
Rails = Badass
DjangoCon 2008
36
Instead?
A boy band?
DjangoCon 2008 37
DjangoCore The Album
DjangoCon 2008
38
DjangoCon 2008
39
Also, a wizard
Simon “Stinky” Willison
DjangoCon 2008 40
Python
DjangoCon 2008 41
Trendy
Google!
DjangoCon 2008 42
But not that trendy
DjangoCon 2008
43
But not that trendy It’s no ruby
DjangoCon 2008
44
But not that trendy It’s no ruby Or Haskell
DjangoCon 2008 45
But not that trendy It’s no ruby Or Haskell Or Erlang
DjangoCon 2008 46
But not that trendy It’s no ruby Or Haskell Or Erlang Or Scala
DjangoCon 2008 47
Significant Whitespace
DjangoCon 2008
48
DjangoCon 2008
49
Salmon
DjangoCon 2008
50
Universe of marshy force. Field sense shallows the hatchery saying Hello,. World!. Hydro. Power spring sometimes; snowmelt powers snowmelt always.
DjangoCon 2008
51
DjangoCon 2008
52
No quantum superpositions
DjangoCon 2008
53
DjangoCon 2008
54
“-”.join(array)
DjangoCon 2008
55
Restarting the server?
WTF!?
DjangoCon 2008 56
WSGIReloadMechanism
DjangoCon 2008 57
“I buy my eggs at the cheese shop”
- Aaron Straup Cope
DjangoCon 2008
58
“I buy my eggs at the cheese shop”
- Aaron Straup Cope
N O O [M
DjangoCon 2008
N A L
] E G A U G
59
DjangoCon 2008
60
__import__()
from future import awesome
DjangoCon 2008
61
Basically, for hippies
Scared of real OO languages
DjangoCon 2008 62
Frameworks
DjangoCon 2008 63
Flamework is a framework
But what does that mean?
DjangoCon 2008
64
Make common tasks:
easier faster error-free
DjangoCon 2008
65
Show a list of photos Write an API method Kill all humans
DjangoCon 2008
66
Front-tier frameworks
Act as glue
DjangoCon 2008 67
Other Services
Data store
Data Model
Business Logic
Request Dispatcher
Interaction Logic
Templates
Framework Users
DjangoCon 2008 68
Great for getting started
Slows you down later?
What is it doing in there?
DjangoCon 2008 69
DjangoCon 2008
70
DjangoCon 2008
71
DjangoCon 2008
72
Fast
Easy
Flexible
DjangoCon 2008
Scalable
73
Large scale is often not a good goal
DjangoCon 2008 74
Skyscrapers are special
DjangoCon 2008
75
Regular houses Don’t need 10 story foundations
DjangoCon 2008
76
Just build!
DjangoCon 2008
77
DjangoCon 2008
78
DjangoCon 2008
79
DjangoCon 2008
80
Everything here is possible!
We care about easy
DjangoCon 2008 81
Large Scale
DjangoCon 2008 82
No multiple DB clusters
DjangoCon 2008
83
Manual partitioning
DjangoCon 2008 84
No multiple hosts per DB
DjangoCon 2008 85
Where do we read from?
DjangoCon 2008
86
How about now?
DjangoCon 2008
87
How about now ?
DjangoCon 2008
88
Read/write consistency
DjangoCon 2008
89
Read/write consistency
LAG!!1
DjangoCon 2008 90
Good for non-writing users
(logged out)
DjangoCon 2008 91
Non time-sensitive data
Feeds Read-only APIs
DjangoCon 2008 92
DjangoCon 2008
93
Cool kids use master/master pairs
DjangoCon 2008 94
Consistent read/write
1. Write to side A 2. Read from side B 3. Replication occurs 4. FAIL
DjangoCon 2008 95
Consistent read/write
1. Write to side A 2. Read from side A 3. ??? 4. PROFIT!!
DjangoCon 2008 96
Then we can start to get fancy!
DjangoCon 2008 97
Where do we read/write?
DjangoCon 2008
98
Abstraction FTW
db_query($cluster_name, $sql)
DjangoCon 2008
99
No sharding
DjangoCon 2008 100
WTF is sharding?
All about books
DjangoCon 2008 101
Dewey Decimal
DjangoCon 2008 102
But it can’t do everything
DjangoCon 2008
103
Makes some things ‘impossible’
No shard walking
DjangoCon 2008 104
DjangoCon 2008
105
Django knows our models
It’s possible
DjangoCon 2008 106
If we accept constraints
DjangoCon 2008
107
No external sequences
DjangoCon 2008 108
Auto-incs can be bad
master/master shards
DjangoCon 2008 109
No automatic denormalization
DjangoCon 2008
110
Counters would be easy!
DjangoCon 2008 111
And copied fields too
DjangoCon 2008
112
High Performance
DjangoCon 2008 113
It’s slow!
DjangoCon 2008
114
DjangoCon 2008
115
DjangoCon 2008
116
There are limits
But they only apply When being really silly
DjangoCon 2008 117
Sessions
DjangoCon 2008
118
OMG UR ON CRAK
DjangoCon 2008 119
Extra DB table
A read on every page
DjangoCon 2008 120
Lightweight = better
DjangoCon 2008
121
Signed cookies
DjangoCon 2008 122
No parallelization
DjangoCon 2008 123
Multiple cache-requests
DjangoCon 2008 124
Multiple multi-keyed reads
DjangoCon 2008
125
Slow HTTP requests while we’re doing other stuff
DjangoCon 2008
126
Creates dumb SQL
DjangoCon 2008
127
foo.exclude(bar = true) WHERE (NOT (bar = 1))
DjangoCon 2008 128
foo.exclude(bar = true) WHERE bar != 1
DjangoCon 2008 129
foo.filter(bar = false) WHERE bar = 0
Indexable!
DjangoCon 2008 130
Maintainability
DjangoCon 2008 131
Maintainability Awesomeness
DjangoCon 2008 132
Verbose template syntax
<h1>{{ foo.bar }}</h1>
DjangoCon 2008 133
<h1>{{ foo.bar }}</h1> <h1>{$foo.bar}</h1> 3 extra characters!
DjangoCon 2008 134
<h1>{{ foo.bar }}</h1> <h1>{$foo.bar|e}</h1> Well almost…
DjangoCon 2008 135
{% if edit_mode %} woo {% endif %}
9 extra!
DjangoCon 2008
{if $edit_mode} woo {/if}
136
Verbose template syntax makes people cry
DjangoCon 2008
137
No query debugger
DjangoCon 2008 138
>>> from django.db import connection >>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_ polls.pub_date FROM polls_polls', 'time': '0.002'}]
DjangoCon 2008
139
>>> from django.db import connection >>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_ polls.pub_date FROM polls_polls', 'time': '0.002'}]
Not good enough!
DjangoCon 2008 140
Better, but not amazing
DjangoCon 2008
141
DjangoCon 2008
142
Page summary
DjangoCon 2008
143
Make it clickable…
DjangoCon 2008
144
EXPLAIN is AWESOME
DjangoCon 2008 145
And/or show the data
DjangoCon 2008
146
MySQL Query Profiler
Enough data to choke a whale
DjangoCon 2008 147
Not smug enough
DjangoCon 2008
148
DjangoCon 2008
149
DjangoCon 2008
150
Unreadable SQL
SELECT `app_userprofile`.`user_id`,`app_userprofile`.`account_status`,`app_userp rofile`.`account_status_reason`,`app_userprofile`.`account_delete_type`,` app_userprofile`.`syndication_key`,`app_userprofile`.`blurb`,`app_userpro file`.`location`,`app_userprofile`.`country`,`app_userprofile`.`postal_code`,` app_userprofile`.`gender`,`app_userprofile`.`theme`,`app_userprofile`.`tiny _photo`,`app_userprofile`.`small_photo`,`app_userprofile`.`smedium_pho to`,`app_userprofile`.`medium_photo`,`app_userprofile`.`large_photo`,`ap p_userprofile`.`orig_photo`,`app_userprofile`.`bday_month`,`app_userpro file`.`bday_day`,`app_userprofile`.`bday_year`,`app_userprofile`.`is_pro`,`a pp_userprofile`.`privacy_settings_id`,`app_userprofile`.`email_settings_id `,`app_userprofile`.`preference_settings_id`,`app_userprofile`.`ad_counte r`,`app_userprofile`.`num_friends`,`app_userprofile`.`num_fans`,`app_user profile`.`num_feeds`,`app_userprofile`.`num_blocked_fans`,`app_userpro file`.`num_friend_requests`,`app_userprofile`.`default_set_id`,`app_userpr ofile`.`invite_id`,`app_userprofile`.`invite_round`,`app_userprofile`.`avail_in vites`,`app_userprofile`.`original_email` FROM `app_userprofile` WHERE (`app_userprofile`.`user_id` = 1)
DjangoCon 2008
151
Unreadable SQL
SELECT * FROM `app_userprofile` WHERE `user_id` = 1
DjangoCon 2008
152
SELECT * is fast! Faster over the wire Faster to parse Easier to debug
DjangoCon 2008 153
Unreadable SQL
SELECT `app_note`.`id`,`app_note`.`type`,`app_note`.`body`,`app_note `.`sender_id`,`app_note`.`date_sent`,`app_note`.`num_recipie nts`,`app_note`.`recipient_type`,`app_note`.`set_id`,`app_not e`.`direct_to_id`,`app_note`.`media_file_id`,`app_note`.`url`,`a pp_note`.`url_inline`,`app_note`.`url_clicks`,`app_note`.`embe d_id`,`app_note`.`event_id`,`app_note`.`is_direct`,`app_note`.` is_public`,`app_note`.`is_deleted`,`app_note`.`is_ad`,`app_no te`.`reply_to_id`,`app_note`.`forwarded_from_id`,`app_note`.` original_note_id`,`app_note`.`stars`,`app_note`.`rsvp` FROM `app_note` LEFT OUTER JOIN `app_userprofile_notes` AS `m2m_app_note__recipients` ON `app_note`.`id` = `m2m_app_note__recipients`.`note_id` WHERE ((NOT (`app_note`.`is_deleted` = True)) AND `m2m_app_note__recipients`.`userprofile_id` = 17573) ORDER BY `app_note`.`date_sent` DESC LIMIT 16
DjangoCon 2008 154
Unreadable SQL
SELECT n.* FROM app_note AS n LEFT OUTER JOIN app_userprofile_notes AS l ON n.id=l.note_id WHERE n.is_deleted = False AND l.userprofile_id=17573 ORDER BY n.date_sent DESC LIMIT 16
DjangoCon 2008 155
Django makes me
scared of joins
DjangoCon 2008 156
Actually, JOINs are evil
DjangoCon 2008 157
DjangoCon 2008
158
Low version number
DjangoCon 2008
159
DjangoCon 2008
160
DjangoCon 2008
161
Suspiciously small
Django: 105 klocs Flamework: 260 klocs
DjangoCon 2008 162
Suspiciously small
Very long lines?
DjangoCon 2008 163
Can’t pluralize octopus
DjangoCon 2008 164
Commit all fields at once
DjangoCon 2008
165
Race conditions!
DjangoCon 2008 166
Dirty flags
DjangoCon 2008 167
Don’t make Malcolm cry
DjangoCon 2008 168
No simple way to inc/dec
DjangoCon 2008
169
UPDATE foo SET x=x+1
Makes consistency hard!
DjangoCon 2008 170
Character encoding Input validation?
DjangoCon 2008
171
Character encoding Input validation?
(Probably broken)
DjangoCon 2008
172
No mascot!?!
DjangoCon 2008
173
Echidna?
DjangoCon 2008
174
MAGICAL POWERS
DjangoCon 2008
175
No deployment system
DjangoCon 2008
176
Extending models
Tables are never big enough
DjangoCon 2008 177
Model changes
manage.py reset appname
DjangoCon 2008
178
django-evolution
dmigrations
DjangoCon 2008 179
What’s django done for you lately?
DjangoCon 2008 180
thanks
(now go make it better)
Photo credits
• • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/wilson/18685396/ flickr.com/photos/uhop/105063089/ flickr.com/photos/plasticbag/91453376/ flickr.com/photos/zenoc/1408162618/ flickr.com/photos/subinev/27810058/ flickr.com/photos/kaptainkobold/2531797176/ flickr.com/photos/esti/243781036/ flickr.com/photos/oneras/253617128/ flickr.com/photos/josefstuefer/72512671/ flickr.com/photos/pgoyette/168076182/ flickr.com/photos/paulhammond/2825869449/ flickr.com/photos/nez/378348754/ flickr.com/photos/calevans/2265863104/ flickr.com/photos/mbiddulph/416190643/ flickr.com/photos/obliterated/2249663391/ flickr.com/photos/salim/67753003/ flickr.com/photos/sosalem/2816909110/ flickr.com/photos/radiorover/324193883/ flickr.com/photos/nickwheeleroz/2773776799/ flickr.com/photos/rse/22972901/ flickr.com/photos/rougerouge/534615363/ flickr.com/photos/hawkexpress/386355092/ flickr.com/photos/escalla/438817719/ flickr.com/photos/mknightphoto/2295688304/ flickr.com/photos/petereed/138369750/
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/sharynmorrow/2571566304/ flickr.com/photos/florathexplora/286035943/ flickr.com/photos/webguru4god/2233500779/ flickr.com/photos/emdot/195189924/ flickr.com/photos/kayepants/391645870/ flickr.com/photos/ppowers/793939285/ flickr.com/photos/dinesh_valke/425540358/ flickr.com/photos/hawkexpress/306276212/ flickr.com/photos/cedric1981/383614033/ flickr.com/photos/macspite/1118042091/ flickr.com/photos/jpstanley/1440357613/ flickr.com/photos/ashclements/248887492/ flickr.com/photos/kecko/1876479840/ flickr.com/photos/difusa/131100622/ flickr.com/photos/amagill/378984660/ flickr.com/photos/bea_k63w-wa/2466009108/ flickr.com/photos/namullim/2096115820/ flickr.com/photos/lotterymonkey/175248733/ flickr.com/photos/13519089@N03/1380483002/ flickr.com/photos/gswj/10344097/ flickr.com/photos/deanj/67607256/ flickr.com/photos/malias/2218694639/ flickr.com/photos/clintjcl/120760767/ flickr.com/photos/grandpamikey/2228239585/ flickr.com/photos/lea/2829752510/ flickr.com/photos/superamit/196481424/ 182
DjangoCon 2008
More photo credits
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/simon999/2419051858/ flickr.com/photos/tonyjcase/2172060723/ flickr.com/photos/heypaul/1428681/ flickr.com/photos/swisscan/2308034084/ flickr.com/photos/apollonia666/310793656/ flickr.com/photos/bdorfman/15846725/ flickr.com/photos/wishymom/566394520/ flickr.com/photos/jgarber/121886004/ flickr.com/photos/eole/380316678/ flickr.com/photos/917press/1940509163/ flickr.com/photos/wallyg/476718069/ flickr.com/photos/biwook/390088839/ flickr.com/photos/bdjsb7/1674027307/ flickr.com/photos/naveenroy/314965221/ flickr.com/photos/wyldkyss/2757474674/ flickr.com/photos/orinrobertjohn/409812627/ flickr.com/photos/candiedwomanire/1651870/ flickr.com/photos/dalvenjah/408497650/ flickr.com/photos/krikit/2657457178/ flickr.com/photos/w5nyv/2370344/ flickr.com/photos/jordanfischer/72510316/ flickr.com/photos/tedsblog/91531710/ flickr.com/photos/celesterc/1069893367/ flickr.com/photos/spacematters/766372252/ flickr.com/photos/esti/147733640/ flickr.com/photos/rbp/2833593771/
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/jordanroher/228435564/ flickr.com/photos/tambako/2524829095/ flickr.com/photos/66132721@N00/2725908636/ flickr.com/photos/freeparking/1279927021/ flickr.com/photos/drurydrama/1352506223/ flickr.com/photos/mn_francis/397620193/ flickr.com/photos/dramaqueennorma/273371415/ flickr.com/photos/chasqui/2338990305/ flickr.com/photos/cityhunter12/2574288725/ flickr.com/photos/akbuthod/1508537255/ flickr.com/photos/nunoforos/2811186142/ flickr.com/photos/marcusjb/432600086/ flickr.com/photos/12495774@N02/2405297371/ flickr.com/photos/larimdame/572477975/ flickr.com/photos/harryharris/1525946537/ flickr.com/photos/davemorris/91808851/ flickr.com/photos/aplumb/121285138/ flickr.com/photos/peebot/825451566/ flickr.com/photos/reillyb/2216541900/ flickr.com/photos/steffe/423086866/ flickr.com/photos/ciordia/255983919/ flickr.com/photos/wwworks/2472230611/ flickr.com/photos/pnoid00/1909096140/ flickr.com/photos/jurvetson/2229899/ flickr.com/photos/foxypar4/766801643/ flickr.com/photos/wizardhere/2413404079/ 183
DjangoCon 2008
these slides are online:
iamcal.com/talks
DjangoCon 2008
184
Cal Henderson
O HAI
DjangoCon 2008
2
My happy face
DjangoCon 2008 3
DjangoCon 2008
4
2.8 billion photos
DjangoCon 2008
5
2.8 billion photos
1 per 2.3 people on earth 9.2 per US resident
DjangoCon 2008 6
35k photos/sec >2m photos/min
DjangoCon 2008
7
Many thousand pages/sec
DjangoCon 2008 8
Scale nerd
DjangoCon 2008
9
Serious Language
DjangoCon 2008 10
Wordpress Yahoo
Mediawiki
DjangoCon 2008
11
Wordpress Yahoo Digg Delicious
Mediawiki
DjangoCon 2008
12
Flamework Serious Toolset
DjangoCon 2008
13
Flamework Flint Flinderbox Flaux Flag
DjangoCon 2008
Flink Flotos Flictionary Flops Fleng
14
Why hate Django?
DjangoCon 2008 15
What does he even know about Django?
DjangoCon 2008
16
Pownce
DjangoCon 2008 17
A fucking blogging engine
DjangoCon 2008
18
Why all the hate?
DjangoCon 2008
19
DjangoCon 2008
20
DjangoCon 2008
21
DjangoCon 2008
22
DjangoCon 2008
23
An attack on several fronts
DjangoCon 2008
24
An attack on several fronts
The team Python Frameworks
DjangoCon 2008 25
Django itself
Performance Scalability Maintainability
DjangoCon 2008 26
The Team
DjangoCon 2008 27
Frameworks
DjangoCon 2008
28
Struts
DjangoCon 2008 29
Craig McClanahan
DjangoCon 2008 30
DjangoCon 2008
31
Exhibit A : Beard
DjangoCon 2008
32
But Struts is for suits!
DjangoCon 2008 33
The Hot Shit
DjangoCon 2008 34
DjangoCon 2008
35
Rails = Badass
DjangoCon 2008
36
Instead?
A boy band?
DjangoCon 2008 37
DjangoCore The Album
DjangoCon 2008
38
DjangoCon 2008
39
Also, a wizard
Simon “Stinky” Willison
DjangoCon 2008 40
Python
DjangoCon 2008 41
Trendy
Google!
DjangoCon 2008 42
But not that trendy
DjangoCon 2008
43
But not that trendy It’s no ruby
DjangoCon 2008
44
But not that trendy It’s no ruby Or Haskell
DjangoCon 2008 45
But not that trendy It’s no ruby Or Haskell Or Erlang
DjangoCon 2008 46
But not that trendy It’s no ruby Or Haskell Or Erlang Or Scala
DjangoCon 2008 47
Significant Whitespace
DjangoCon 2008
48
DjangoCon 2008
49
Salmon
DjangoCon 2008
50
Universe of marshy force. Field sense shallows the hatchery saying Hello,. World!. Hydro. Power spring sometimes; snowmelt powers snowmelt always.
DjangoCon 2008
51
DjangoCon 2008
52
No quantum superpositions
DjangoCon 2008
53
DjangoCon 2008
54
“-”.join(array)
DjangoCon 2008
55
Restarting the server?
WTF!?
DjangoCon 2008 56
WSGIReloadMechanism
DjangoCon 2008 57
“I buy my eggs at the cheese shop”
- Aaron Straup Cope
DjangoCon 2008
58
“I buy my eggs at the cheese shop”
- Aaron Straup Cope
N O O [M
DjangoCon 2008
N A L
] E G A U G
59
DjangoCon 2008
60
__import__()
from future import awesome
DjangoCon 2008
61
Basically, for hippies
Scared of real OO languages
DjangoCon 2008 62
Frameworks
DjangoCon 2008 63
Flamework is a framework
But what does that mean?
DjangoCon 2008
64
Make common tasks:
easier faster error-free
DjangoCon 2008
65
Show a list of photos Write an API method Kill all humans
DjangoCon 2008
66
Front-tier frameworks
Act as glue
DjangoCon 2008 67
Other Services
Data store
Data Model
Business Logic
Request Dispatcher
Interaction Logic
Templates
Framework Users
DjangoCon 2008 68
Great for getting started
Slows you down later?
What is it doing in there?
DjangoCon 2008 69
DjangoCon 2008
70
DjangoCon 2008
71
DjangoCon 2008
72
Fast
Easy
Flexible
DjangoCon 2008
Scalable
73
Large scale is often not a good goal
DjangoCon 2008 74
Skyscrapers are special
DjangoCon 2008
75
Regular houses Don’t need 10 story foundations
DjangoCon 2008
76
Just build!
DjangoCon 2008
77
DjangoCon 2008
78
DjangoCon 2008
79
DjangoCon 2008
80
Everything here is possible!
We care about easy
DjangoCon 2008 81
Large Scale
DjangoCon 2008 82
No multiple DB clusters
DjangoCon 2008
83
Manual partitioning
DjangoCon 2008 84
No multiple hosts per DB
DjangoCon 2008 85
Where do we read from?
DjangoCon 2008
86
How about now?
DjangoCon 2008
87
How about now ?
DjangoCon 2008
88
Read/write consistency
DjangoCon 2008
89
Read/write consistency
LAG!!1
DjangoCon 2008 90
Good for non-writing users
(logged out)
DjangoCon 2008 91
Non time-sensitive data
Feeds Read-only APIs
DjangoCon 2008 92
DjangoCon 2008
93
Cool kids use master/master pairs
DjangoCon 2008 94
Consistent read/write
1. Write to side A 2. Read from side B 3. Replication occurs 4. FAIL
DjangoCon 2008 95
Consistent read/write
1. Write to side A 2. Read from side A 3. ??? 4. PROFIT!!
DjangoCon 2008 96
Then we can start to get fancy!
DjangoCon 2008 97
Where do we read/write?
DjangoCon 2008
98
Abstraction FTW
db_query($cluster_name, $sql)
DjangoCon 2008
99
No sharding
DjangoCon 2008 100
WTF is sharding?
All about books
DjangoCon 2008 101
Dewey Decimal
DjangoCon 2008 102
But it can’t do everything
DjangoCon 2008
103
Makes some things ‘impossible’
No shard walking
DjangoCon 2008 104
DjangoCon 2008
105
Django knows our models
It’s possible
DjangoCon 2008 106
If we accept constraints
DjangoCon 2008
107
No external sequences
DjangoCon 2008 108
Auto-incs can be bad
master/master shards
DjangoCon 2008 109
No automatic denormalization
DjangoCon 2008
110
Counters would be easy!
DjangoCon 2008 111
And copied fields too
DjangoCon 2008
112
High Performance
DjangoCon 2008 113
It’s slow!
DjangoCon 2008
114
DjangoCon 2008
115
DjangoCon 2008
116
There are limits
But they only apply When being really silly
DjangoCon 2008 117
Sessions
DjangoCon 2008
118
OMG UR ON CRAK
DjangoCon 2008 119
Extra DB table
A read on every page
DjangoCon 2008 120
Lightweight = better
DjangoCon 2008
121
Signed cookies
DjangoCon 2008 122
No parallelization
DjangoCon 2008 123
Multiple cache-requests
DjangoCon 2008 124
Multiple multi-keyed reads
DjangoCon 2008
125
Slow HTTP requests while we’re doing other stuff
DjangoCon 2008
126
Creates dumb SQL
DjangoCon 2008
127
foo.exclude(bar = true) WHERE (NOT (bar = 1))
DjangoCon 2008 128
foo.exclude(bar = true) WHERE bar != 1
DjangoCon 2008 129
foo.filter(bar = false) WHERE bar = 0
Indexable!
DjangoCon 2008 130
Maintainability
DjangoCon 2008 131
Maintainability Awesomeness
DjangoCon 2008 132
Verbose template syntax
<h1>{{ foo.bar }}</h1>
DjangoCon 2008 133
<h1>{{ foo.bar }}</h1> <h1>{$foo.bar}</h1> 3 extra characters!
DjangoCon 2008 134
<h1>{{ foo.bar }}</h1> <h1>{$foo.bar|e}</h1> Well almost…
DjangoCon 2008 135
{% if edit_mode %} woo {% endif %}
9 extra!
DjangoCon 2008
{if $edit_mode} woo {/if}
136
Verbose template syntax makes people cry
DjangoCon 2008
137
No query debugger
DjangoCon 2008 138
>>> from django.db import connection >>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_ polls.pub_date FROM polls_polls', 'time': '0.002'}]
DjangoCon 2008
139
>>> from django.db import connection >>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_ polls.pub_date FROM polls_polls', 'time': '0.002'}]
Not good enough!
DjangoCon 2008 140
Better, but not amazing
DjangoCon 2008
141
DjangoCon 2008
142
Page summary
DjangoCon 2008
143
Make it clickable…
DjangoCon 2008
144
EXPLAIN is AWESOME
DjangoCon 2008 145
And/or show the data
DjangoCon 2008
146
MySQL Query Profiler
Enough data to choke a whale
DjangoCon 2008 147
Not smug enough
DjangoCon 2008
148
DjangoCon 2008
149
DjangoCon 2008
150
Unreadable SQL
SELECT `app_userprofile`.`user_id`,`app_userprofile`.`account_status`,`app_userp rofile`.`account_status_reason`,`app_userprofile`.`account_delete_type`,` app_userprofile`.`syndication_key`,`app_userprofile`.`blurb`,`app_userpro file`.`location`,`app_userprofile`.`country`,`app_userprofile`.`postal_code`,` app_userprofile`.`gender`,`app_userprofile`.`theme`,`app_userprofile`.`tiny _photo`,`app_userprofile`.`small_photo`,`app_userprofile`.`smedium_pho to`,`app_userprofile`.`medium_photo`,`app_userprofile`.`large_photo`,`ap p_userprofile`.`orig_photo`,`app_userprofile`.`bday_month`,`app_userpro file`.`bday_day`,`app_userprofile`.`bday_year`,`app_userprofile`.`is_pro`,`a pp_userprofile`.`privacy_settings_id`,`app_userprofile`.`email_settings_id `,`app_userprofile`.`preference_settings_id`,`app_userprofile`.`ad_counte r`,`app_userprofile`.`num_friends`,`app_userprofile`.`num_fans`,`app_user profile`.`num_feeds`,`app_userprofile`.`num_blocked_fans`,`app_userpro file`.`num_friend_requests`,`app_userprofile`.`default_set_id`,`app_userpr ofile`.`invite_id`,`app_userprofile`.`invite_round`,`app_userprofile`.`avail_in vites`,`app_userprofile`.`original_email` FROM `app_userprofile` WHERE (`app_userprofile`.`user_id` = 1)
DjangoCon 2008
151
Unreadable SQL
SELECT * FROM `app_userprofile` WHERE `user_id` = 1
DjangoCon 2008
152
SELECT * is fast! Faster over the wire Faster to parse Easier to debug
DjangoCon 2008 153
Unreadable SQL
SELECT `app_note`.`id`,`app_note`.`type`,`app_note`.`body`,`app_note `.`sender_id`,`app_note`.`date_sent`,`app_note`.`num_recipie nts`,`app_note`.`recipient_type`,`app_note`.`set_id`,`app_not e`.`direct_to_id`,`app_note`.`media_file_id`,`app_note`.`url`,`a pp_note`.`url_inline`,`app_note`.`url_clicks`,`app_note`.`embe d_id`,`app_note`.`event_id`,`app_note`.`is_direct`,`app_note`.` is_public`,`app_note`.`is_deleted`,`app_note`.`is_ad`,`app_no te`.`reply_to_id`,`app_note`.`forwarded_from_id`,`app_note`.` original_note_id`,`app_note`.`stars`,`app_note`.`rsvp` FROM `app_note` LEFT OUTER JOIN `app_userprofile_notes` AS `m2m_app_note__recipients` ON `app_note`.`id` = `m2m_app_note__recipients`.`note_id` WHERE ((NOT (`app_note`.`is_deleted` = True)) AND `m2m_app_note__recipients`.`userprofile_id` = 17573) ORDER BY `app_note`.`date_sent` DESC LIMIT 16
DjangoCon 2008 154
Unreadable SQL
SELECT n.* FROM app_note AS n LEFT OUTER JOIN app_userprofile_notes AS l ON n.id=l.note_id WHERE n.is_deleted = False AND l.userprofile_id=17573 ORDER BY n.date_sent DESC LIMIT 16
DjangoCon 2008 155
Django makes me
scared of joins
DjangoCon 2008 156
Actually, JOINs are evil
DjangoCon 2008 157
DjangoCon 2008
158
Low version number
DjangoCon 2008
159
DjangoCon 2008
160
DjangoCon 2008
161
Suspiciously small
Django: 105 klocs Flamework: 260 klocs
DjangoCon 2008 162
Suspiciously small
Very long lines?
DjangoCon 2008 163
Can’t pluralize octopus
DjangoCon 2008 164
Commit all fields at once
DjangoCon 2008
165
Race conditions!
DjangoCon 2008 166
Dirty flags
DjangoCon 2008 167
Don’t make Malcolm cry
DjangoCon 2008 168
No simple way to inc/dec
DjangoCon 2008
169
UPDATE foo SET x=x+1
Makes consistency hard!
DjangoCon 2008 170
Character encoding Input validation?
DjangoCon 2008
171
Character encoding Input validation?
(Probably broken)
DjangoCon 2008
172
No mascot!?!
DjangoCon 2008
173
Echidna?
DjangoCon 2008
174
MAGICAL POWERS
DjangoCon 2008
175
No deployment system
DjangoCon 2008
176
Extending models
Tables are never big enough
DjangoCon 2008 177
Model changes
manage.py reset appname
DjangoCon 2008
178
django-evolution
dmigrations
DjangoCon 2008 179
What’s django done for you lately?
DjangoCon 2008 180
thanks
(now go make it better)
Photo credits
• • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/wilson/18685396/ flickr.com/photos/uhop/105063089/ flickr.com/photos/plasticbag/91453376/ flickr.com/photos/zenoc/1408162618/ flickr.com/photos/subinev/27810058/ flickr.com/photos/kaptainkobold/2531797176/ flickr.com/photos/esti/243781036/ flickr.com/photos/oneras/253617128/ flickr.com/photos/josefstuefer/72512671/ flickr.com/photos/pgoyette/168076182/ flickr.com/photos/paulhammond/2825869449/ flickr.com/photos/nez/378348754/ flickr.com/photos/calevans/2265863104/ flickr.com/photos/mbiddulph/416190643/ flickr.com/photos/obliterated/2249663391/ flickr.com/photos/salim/67753003/ flickr.com/photos/sosalem/2816909110/ flickr.com/photos/radiorover/324193883/ flickr.com/photos/nickwheeleroz/2773776799/ flickr.com/photos/rse/22972901/ flickr.com/photos/rougerouge/534615363/ flickr.com/photos/hawkexpress/386355092/ flickr.com/photos/escalla/438817719/ flickr.com/photos/mknightphoto/2295688304/ flickr.com/photos/petereed/138369750/
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/sharynmorrow/2571566304/ flickr.com/photos/florathexplora/286035943/ flickr.com/photos/webguru4god/2233500779/ flickr.com/photos/emdot/195189924/ flickr.com/photos/kayepants/391645870/ flickr.com/photos/ppowers/793939285/ flickr.com/photos/dinesh_valke/425540358/ flickr.com/photos/hawkexpress/306276212/ flickr.com/photos/cedric1981/383614033/ flickr.com/photos/macspite/1118042091/ flickr.com/photos/jpstanley/1440357613/ flickr.com/photos/ashclements/248887492/ flickr.com/photos/kecko/1876479840/ flickr.com/photos/difusa/131100622/ flickr.com/photos/amagill/378984660/ flickr.com/photos/bea_k63w-wa/2466009108/ flickr.com/photos/namullim/2096115820/ flickr.com/photos/lotterymonkey/175248733/ flickr.com/photos/13519089@N03/1380483002/ flickr.com/photos/gswj/10344097/ flickr.com/photos/deanj/67607256/ flickr.com/photos/malias/2218694639/ flickr.com/photos/clintjcl/120760767/ flickr.com/photos/grandpamikey/2228239585/ flickr.com/photos/lea/2829752510/ flickr.com/photos/superamit/196481424/ 182
DjangoCon 2008
More photo credits
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/simon999/2419051858/ flickr.com/photos/tonyjcase/2172060723/ flickr.com/photos/heypaul/1428681/ flickr.com/photos/swisscan/2308034084/ flickr.com/photos/apollonia666/310793656/ flickr.com/photos/bdorfman/15846725/ flickr.com/photos/wishymom/566394520/ flickr.com/photos/jgarber/121886004/ flickr.com/photos/eole/380316678/ flickr.com/photos/917press/1940509163/ flickr.com/photos/wallyg/476718069/ flickr.com/photos/biwook/390088839/ flickr.com/photos/bdjsb7/1674027307/ flickr.com/photos/naveenroy/314965221/ flickr.com/photos/wyldkyss/2757474674/ flickr.com/photos/orinrobertjohn/409812627/ flickr.com/photos/candiedwomanire/1651870/ flickr.com/photos/dalvenjah/408497650/ flickr.com/photos/krikit/2657457178/ flickr.com/photos/w5nyv/2370344/ flickr.com/photos/jordanfischer/72510316/ flickr.com/photos/tedsblog/91531710/ flickr.com/photos/celesterc/1069893367/ flickr.com/photos/spacematters/766372252/ flickr.com/photos/esti/147733640/ flickr.com/photos/rbp/2833593771/
• • • • • • • • • • • • • • • • • • • • • • • • • •
flickr.com/photos/jordanroher/228435564/ flickr.com/photos/tambako/2524829095/ flickr.com/photos/66132721@N00/2725908636/ flickr.com/photos/freeparking/1279927021/ flickr.com/photos/drurydrama/1352506223/ flickr.com/photos/mn_francis/397620193/ flickr.com/photos/dramaqueennorma/273371415/ flickr.com/photos/chasqui/2338990305/ flickr.com/photos/cityhunter12/2574288725/ flickr.com/photos/akbuthod/1508537255/ flickr.com/photos/nunoforos/2811186142/ flickr.com/photos/marcusjb/432600086/ flickr.com/photos/12495774@N02/2405297371/ flickr.com/photos/larimdame/572477975/ flickr.com/photos/harryharris/1525946537/ flickr.com/photos/davemorris/91808851/ flickr.com/photos/aplumb/121285138/ flickr.com/photos/peebot/825451566/ flickr.com/photos/reillyb/2216541900/ flickr.com/photos/steffe/423086866/ flickr.com/photos/ciordia/255983919/ flickr.com/photos/wwworks/2472230611/ flickr.com/photos/pnoid00/1909096140/ flickr.com/photos/jurvetson/2229899/ flickr.com/photos/foxypar4/766801643/ flickr.com/photos/wizardhere/2413404079/ 183
DjangoCon 2008
these slides are online:
iamcal.com/talks
DjangoCon 2008
184
