A minimal model for fast scrambling