Saturday, November 17, 2007

An old talk by Verner Vogels on Distributed Systems

Long ago, Amazon Bangalore held an event where Verner Vogels had been invited to speak on designing distributed systems on a planetary scale. I had taken notes which surfaced when cleaning up the house. I decided to put them up online:

1. Use scalable primitives (RPC breakable easily)
2. Cache near the edges
3. Hierarchies and functional partitioning
4. Use aggregation, data fusion
5. Do not conceal Heterogenity
6. Be strict in what you emit, liberal in what you accept.
7. Avoid strong consistency properties
-Never expect your system to be stable
-Assume that nodes are leaving, joining, failing
Control:
For control to work, the system needs to be deterministic (hard)
-Apply a top down approach to controlling
-Cannot use force to put them into a model.
-"Real life in essence is probabilistic". Let go of Control

Self organizing systems:
-Positive feedback, or negative feedback

Robustness in Biologicial systems
---------------------------------

-Redendancy
-Feedback
-Modularity
-Loose coupling
-purging
-Apoptosis (programmed cell death. 50-70 billion cells commit suicide)
-Spatial compartmentalization
-Extended Phenotype
Scaling the organization
------------------------
-Organization needs to be bottom up.
-Functional units need to behave like organisms, can take care of themselves.
-Nodes recycle all the time
-Reboot becomes a tool
-Stability of organism is key, even if cell des

Continuous introspection:
Nodes responsible for themselves, not outside monitoring

The power of Epidemics
----------------------
Probabilistic model: eventual consistency
A synchronous communication pattern
Autonomous and decentralized actions
Robust with respect to message loss/node failure
Rigorous mathematical underpinnings

Epidemic algorithms and protocols:

Choose a random subset of operations
2 phases:
Phase I:
1-> N/2
Initial rate of growth factor of 2
Half way factor of 1.4
Near end factor of 1
Phase II:
nearly all nodes infected
O(logn) # rounds needed to infect entire population

Failure detection Service:
Local for a last update to a node's site
If timestamp is not update, you know of disconnection
Probabilistic, reliable
-buffer received messages
-garbage collection suffers from scalability problem

Distributed State Maintenance:
-------------------------------
State Engine: a distributed database table
Leaves are like rows
-lives on net
Randomized Rumour spreading
use:
-autonomous , asynchronous behaviour
-Let go of control, deterministic techniques

Saturday, October 06, 2007

Randy Pausch and Ratatouille

I happened to come across Randy Pausch and Ratatouille at the same time. I find a connection between the two- Randy talks of achieving your childhood dreams, and Ratatouille is about the rat who goes on to live his dream of becoming a chef. Randy talks about brick walls that were meant to keep the "other" people out who did not want something badly enough, and in Ratatouille, the chef Gusteau reminds Remy not to dwell on the past but to go up and look around.

Ratatouille is an amazing engineering achievement. Full global illumination! I got inspired enough to order Matt Phar's Rhysically based Rendering - I've been eyeing the book for over a year, but with the dollar falling, it no longer seems expensive. Lets see if I can render skin the way it looks in the film.

Monday, August 20, 2007

Salsa:Corrective Dancing tips

Corrective Dancing tips from Eddie the Salsa freak:

These were tips given out by Eddie at the Congresss this weekend, to correct mistakes.

1) Mens Posture: Put both hands over your head. They form a frame. Bring the hands down, in front, keeping that same frame intact. Your elbows should be 6 -7 inches forward (if measured from your back ). Imagine having two oranges under your armpits, so that your arms extend away from your body rather than stay close.

2)Ladies right spin: Imagine a pizza around the ladies head, men should not invade the space inside when turning. Beats 1,2,3 is for preparing the lady for a turn. If you are doing 3,4 turns, then you prepare for a turn on beat 2. During beats 5,6,7 , men should NOT step back as then you will have your hands over the ladies head. step sideways or just behind your left shoe, at an angle.
3)Spinning Fingers: Use multiple fingers rather than one for spinning.

4)Turns wil have better momentum when you use your hands to generate momentum. Keep them near your centre of gravity , which is just below chest for men and lower down for women.

4)Loose arms will make it hard for a lady to be led. Moving the arm will not move the body and so the lead cannot be followed.

Tips from other instructors:

5)Cross Body lead : Normally you open up on beat 3. However, if you start to turn your body leftward on 2, the lady has a much better idea that a cross body lead is coming, and so the men have more time on beat three to think about the next move rather than having to push the lady to turn.

6) The direction of feet can indicate the next coming move , so do not always assume you have to end a move with both feet together. The left feet in front pointed to the right indicates the man will do a left turn soon.

Friday, August 10, 2007

At the memorial

Stay behind until dark-
It is peaceful and beautiful here at night,
and lying down on this field,
you can count the stars and
feel the wind across the Potomac
on your face.
Not interested ?
My fault perhaps,for when you're 21
small things seems miraculous and magical,
and youth expendable, on such poetic indulgences.
But you who stand beside me now,
middle aged,mediocre,but survivor anyhow,
peering at crosses that stretch across the plain
reading rank,medals and name-
you were always wiser,to not have enlisted.
For I'm 21, six feet under, and dead.

Wednesday, August 08, 2007

The Lonely wife

I confess...
I've never tried the internet before..
I am a little nervous, but curious even more.
woman in mid thirties,accomplished, successful,
Wants time out with someone playful,
I am married, feel supported, quite respected,
but at times ignored,somewhat neglected.
My fingers tremble, type type type..
But I've heard the voices inside
need a little fun on the side.

Sunday, July 08, 2007

The thief

All week I wait
for a glimpse of those eyes.
They do not know the madness
that lives in me.
Hidden behind the civil smile,
we will talk for a while
of future plans and possiblities
While a scheming fuguitive on the run
breaks into the room in her heart,
but empty handed, departs.

Another week I wait....
Time flies fast,
Days burn in anticipation in the sun
and wither like dry flowers in the night.
The voices of reason have long since drowned
Drunk in the gaze of those eyes.

Saturday, June 23, 2007

The Art of forgetting

Someday I shall certainly forget you,
Like the grass in the fields forgets
the winter snow
Like the fog of winter
leaves the autumn sunsets only a memory.

On solitary Sunday afternoons,
My steps take the old familiar road
A flight of stairs, a familiar song,
Tapping feet on a wooden floor.

Yes I shall certainly forget you,
in this crowd of beautiful faces,
whose names within a moment of knowing
I may forget to recall
Weren't you just another among them all?

And yet sometimes I just might,
staring into a pair of dark eyes
(seems like yours, in disguise ?)
Or looking at a baby's familiar smile,
remember you for a while.

As I weave dance patterns on the floor
if a whiff of that familiar perfume
were to drift across the room
I might lose my step and stall
(while my partner stares, appalled)


Tossed by the wind across the street
A small black butterfly
caresses my cheeks
The memory of you, and reminder to
the ephemeral finality of life itself.

Someday with a peaceful smile
I can forget to forget you.
Like a lonely sea that remembers
to return everything to the shore.

Wednesday, June 20, 2007

The old house

A short story based on the Jal song 'Aadat'




The day was hot and I had been selling insurance all day, so the empty bench in the garden of the old house made me decide to rest a while. This was the outskirts of the city and I still had a few clients in the area to cover by nightfall.
It had been a nice house in its time- ornate engravings on the walls, a large fountain in the garden and the large arched windows of stained glass told that its original inhabitants had been wealthy. But later owners had constructed a crude wood staircase that angled its way down outside the house and spoilt the look completely. Most of the windows were grilled up; the fountain was covered with weeds. A dead tree arched its way up towards the first floor balcony. Wild, but beautiful flowers grew in the garden, and the birds were singing. I decided on a fifteen minute break.

I must have slept off for an hour. The late afternoon sun had disappeared behind the house. I was awakened by the sound of running water near me. Somebody had switched on the fountains and the pipes in the garden. It was not an abandoned house after all- I was trespassing, and before someone came out to tell me to get the hell out, I reached for my briefcase and stood up to leave.

But I had been noticed already. An outline of a figure appeared behind the large first floor windows, and then, from the shadows of the first floor balcony, the prettiest face in the world looked down. She was dressed in white and her wavy hair ran till her waist. “Lost your way around town, stranger?” she smiled. “Maybe you could use my help.” She started walking down the large wooden stairs.

“I did not mean to- um, had no idea... I was just going to leave” I said apologetically, and prepared myself for a lecture on civic sense.

“That’s all right,” she said as she came and stood next to me. “We don’t get many visitors here anyways, out here it’s only me and Pa, so it’s good to see a new face and hear news from the town. Why don’t you come on in for some lemonade before you head back?”

She smelt of Jasmine. She must have been around twenty, and whatever initial hesitation I had succumbed when she gently tapped her hand my shoulder. Her touch was cool, and before I knew it I was following up the stairs.

On the first floor an old man with a beard lay fast asleep on a rocking chair, with a newspaper folded on his lap. The girl stopped, placed her finger on her lips to tell me not to disturb him. She turned right, and I followed her into her room. It was dimply lit,with beautiful, ornate furniture but a bit old fashioned, like the kind they used to have fifty years ago. A luxurious beige leather sofa and a queen sized bed took up most of the room. An upright piano and a bookshelf full of sheet music filled up the rest. A portrait of the room’s owner hung on the walls, next to a clock that was no longer working.

“The place looks classy, doesn’t it… but one could find it a bit spooky at times,“ she turned and giggled.
I found myself recalling the Beatles song in my mind. “I once had a girl, or should I say, she once had me…She showed me her room….

“I’ll go get your lemonade,” she said. “Lots of ice, I assume? Its been a hot day for you?”
I nodded, and she disappeared through a door.
When she returned, she placed the jug and two glasses at the table next to the sofa and hopped in to bed and giggled again.

“So what brings you here?” she asked.
“I sell insurance. It’s a traveling job. I move around from town to town.”
“Really? How exciting… and we have been here at the same house for years.
“Yes, I like moving around, seeing new places, like this town.” I poured myself some lemonade.

“Pa does not let me go out. It’s forbidden, and has been the way since mother died. I can step out for a bit in the evenings. Or like now, when he’s asleep.

“Maybe he’s very protective of you,” I suggested. “All the young men from college vying for your attention”

“College stopped two years ago,” she said. “I was studying music. But last when I went and checked, they had taken my name off the rolls. So now it’s me and my piano, most of the time.”
“As for young men, there was one- a face a bit like yours, to be honest. It was love at first sight. But I lost him.” She looked out of the window at the dying sun and sighed.

I tried some encouragement. “Life goes on” I said. “Good things will happen,” A part of me was telling me to get back to work, and not delve into the private lives of strangers. The other part was falling in love with her.

“True, life goes on,” she said. “But I don’t, its’ as if the world has stopped.” She paused.
“I used to play a song for him you know, on the piano,” she said. “I know you’ll find this really strange, but would you like to hear a bit of it? It’ll make me very happy.” She looked at me expectantly with her large beautiful eyes.

“I would love to, “I said. I would have given the world to please her.

She tiptoed and closed the door and took out a notebook from the shelf and placed it over the piano. The music started. Halfway though the song I walked across the room to stand next to her. She was in tears when she finished. I patted her on the shoulder. She held my hand and would not let go.
“You’ll better get going stranger,” she finally said. “I’ll walk you down. “

She opened another door and led me down a spiral staircase that was behind the house.
The chill night breeze greeted us. “I’ll be around these parts again in two weeks, “I said as we stepped down. “I wish we could meet again sometime”.

“Very well, we shall see. It’s hard to catch up with me. You might see me some days,” she smiled.

“There is one interesting thing about this house that I must show you though,” she said as we were walking out through the garden behind. “This garden has a well, a wishing well. You close your eyes and make a wish. Then you drop a coin. Within a month, your wish comes true.” She pointed out the well to our right. It was about five feet wide with a two feet wall along its periphery.

“Really? “ I felt charmed by her childlike conviction.
“Yes. Come, we will try it now.”
We walked over to the well and I took out a coin from my pocket.
“Well now close your eyes.” She said innocently.

I was about to close my eyes when the lights in the house came on, and someone shouted from behind.
“Stop. Don’t do it.” A loud stern voice came from behind.

It was the old man, her father. He stood at the rear door with a handgun pointed at me.
“Get out of here right now, if you value your life” he repeated, with fear and anger on his face as he rushed towards me.

The girl next to me clutched my hand and started screaming. “Its unfair of you Pa, you never let me have any friends. Please Pa…”

“Step back from the well”, the old man said icily,” if you don’t want her to pull you down with her. Step back or I shoot“

“All right… please don’t shoot.” Too shocked and dazed to say anything, I let go of the hand of my new friend and started stepping away. “I will be back” I whispered to her. An overprotective father, and a bit senile too, I thought.

“Forget this night, forget all that may have been said,” the old man said, as he followed me with his handgun towards the exit. “Please don’t try coming here again, and its a warning.” He followed me out to the street and to my car. As I drove away, I could still see my new friend standing in the garden. I have to come back, I told myself.
______________________________________________________________

Date : 15 June, 2002

To:
General Manager Life Policies,
Eastern Region,
Global Insurance Corporation
Karachi.
Sub: Report on disappearance of sales representative

Dear Sir,

We, Hidden Eye Detectives, have been engaged by your company to track the sudden disappearance of your agent Mr. Khan in Lahore with large sums of collected payments in his possession (case no 1332KH). We are pleased to report we have traced Mr. Khan within a week of your request. The missing amount in question is safe and Mr. Khan appears to have had no intentions to disappear with it. His behavior however, is peculiar and suggests some mental disturbance, more of which is noted below.

Our agent traced MR. Khan to a hotel in the area and followed him every day to the outskirts of an abandoned house in Lahore. The house’s lone occupant, and old businessman who lost his daughter in an accident years ago, had died a few weeks ago after a fall from a staircase. Mr. Khan was found sitting on a bench in front of the house every day till dusk, occasionally talking to himself. He had no interest in the collected payments in his possession, which he gladly handed over to us from his hotel room. He seems to have not bothered to stay in touch with his relatives as well.

Our investigation is concluded, we are returning the collected amounts in person tomorrow. We do not feel criminal investigations against Mr. Khan are required.
Yours truly,
Obaid R.
Hidden Eye Detectives.


Date: 16 June 2002

To:
General Manager Life Policies,
Eastern Region,
Global Insurance Corporation
Karachi.

Sub: Report on disappearance of sales representative (Update)

Dear Sir,
We regret to say; we were just informed by the local police that Mr. Khan was found dead in a well in the grounds of the abandoned house he frequented. A few cases of people losing their way at night and falling into an open well on the premises have been reported in the past, so no foul play is suspected.
The collected amounts are being sent today.
Yours truly,
Obaid R.
Hidden Eye Detectives.

Sunday, April 08, 2007

The Namesake

This is a film filled with great compassion and tenderness...and like before, Mira Nair manages to create beautiful shots out of run down urban Indian cityscapes. I do find the ending weak and unresolved-but the flaw is inherited from the original novel, so more of this later.
Ifran Khan and Tabu's arranged marriage and subsequent relocation to Massachusetts, the slowly growing intimacy between them in a lonely land, is captured beautifully in an almost black and white shot of the wintry Boston landscape- on a snowed out grey morning, Ifran climbs the snow covered stairs and waves goodbye to Ashima standing inside the apartment, whose raised hand is silhouetted against the snow outside . The part of the film evokes memories of Rays 'Apu's World' in its approach to showing intimacy.

Irfan and Tabu's quiet and subdued acting is extraordinary and natural, the births and deaths in the family, the growing up of their teenage children, move the story along. The second half is about their son Gogol, who changes his name in a small act of rebellion against his traditional parents. Gogol's white girlfriend plays a small stereotypical cameo and exits. Unlike their immigrant parents, the US born children have never had to undergo any real suffering- perhaps this is why the relationships in the second half lack intensity compared to the first. The story ends predictably with Tabus eventual return to Calcutta, but does not really resolve Gogol's future. The film makes the best out of a book whose theme of the immigrant experience, has been much explored. Also, the book may have been about Gogol, but the film had already reached its pinnacle in the first half, Gogol's life story somehow cannot compare to it.

Friday, March 02, 2007

A Star is born

A report on an American Idol style contest in Bangladesh .

There were over fifty thousand contestants,99 of whom would make it to the final round in Dhaka, to participate in Bangladesh's version of American idol. Among the many videos of this contest uploaded on you tube is one of a very ordinary looking,you could say 'rustic' schoolgirl,singing in the first elimination round. After the first few lines of her first song, the judges ask her to sing something contemporary. Its not that impressive. Then Ahmed Imtiaz Bulbul, a a film music composer in his country of the same stature as AR Rehman, asks her for a folk song. She sings in a strong voice far mature beyond her age, with a rhytm and intensity that has the judges clapping on the table with her. She is through to the finals.

Though other small interviews of the contestants and their families, we learn that she is poor, she's not had as much training as the others, that she only sings folk songs of Lalon, a folk singer and spititual leader of 19th century Bengal. She talks of how she would listen to famous singers on cassettes (no CD player) and try to emulate them. Her father appears in an interview in a very rural setting.

The second round of the finals passes by- nothing noticeably differnt, but she makes it to the next level. She is the youngest competitor.In the customary taped message for round 3, she appears in a red dress and a cute voice says:
"I'm Salma Akhter, I'm from the province where Lalon was born....I've faced lots of hardship while learning to sing. Please vote for me..."

On stage, we see the transformation- a pretty face and a newly acquired stage presence. Judge Imtiaz Bulbul tells her after the performance: "Someday when I give up this life and become a bard, I will come to your house and ask for three things-will you oblige ?" Salma nods. "First, I would want a meal. Next,shelter for a night, and third, grant the wish to be born as your child.". Bulbul,like the rest of his national audience,is getting swayed by her charm. Salma advances through the rounds, on the basis of huge audience votes. Bulbul will occasionally burst out in songs in her praise throughout the rounds.

Competition thickens,comments on you tube postings of the rounds point out how she can sing only folk, how judge Bulbul is obsessed about getting her married to his son. Expatriates comment on whether a modern singer would better represent the nation than this contry girl. Some other notable contestants appear, Muhim, Nishita and Ronty. The latter
is extremely versatile ,perhaps the most talented ,and she seems to get the highest judge
ratings, but only a half of the votes that Salma gets. She will eventually exit in the penultimate round, after having sung a few haunting numbers. Salma starts to get loads of fans calling on her cell saying they appreciate her music.

At the end, four are left- three of whom will make it to the final.Every contestant, including Salma,believes that she will be the one to exit- but she makes it. The last round is in December 2006. Among the three contestants left, the results are no surprise.
After Salma's song, Bulbul makes a strange comment about how you can smell milk near children, and how 'we get the same smell of newness in you'. She wins a car,lakhs of rupees from the sponsor, and a music tour of US, UK and Canada. Where will she go from here ? Throughout the contest,the top finalists appeared oblivious of their fame, though they had become familiar faces among the international diaspora. It has not yet gone to their heads.

Sunday, February 18, 2007

new year resolutions

(Yes, some people make them early , and I take a whole month to think what they should be)

1.To be more organized :
a) at work : manage email better
b) at home ( a more systematic method for cleaning the house, buying groceries)
c) life in general: plan before doing, anticipate

2. To finish things:
a) Finish what I take up, including the short film.
b)Not take up too many things, nor get drawn to multiple things.

3. To be focused, and not distract myself when doing one thing.
Specially not internet.

Technical things to consider learning:
PHP 5, Lisp, Natural phenomena in realtime hardware with GLSL
(but only a few of this list; only after the film; and only 1 at a time)

Monday, January 15, 2007

Books I read in 2006

  1. Lahore
  2. Maximum City
  3. Punjabi Century
  4. Passion India
  5. Do and Die
  6. That summer in Paris
  7. Decision making
  8. Built to last
  9. Too soon old too late smart
  10. Effective C++

Half read:OpenGL shading language

Read a few chapters:
Effective STL
Rapid development
Learning Perl