Federal regulators want to impose new guardrails on the way retail investment firms such as Robinhood Markets use advanced analytics to encourage customers to trade, the latest in a series of policy efforts prompted by the 2021 meme-stock craze.