#spark pipeline monitoring