Zillow had staked its future growth on its digital home-flipping business, but getting the algorithm right proved difficult. The WSJ digs in and explores what forced it to backpedal.