Albert Costa et.al. say yes:
Should you sacrifice one man to save five? Whatever your answer, it should not depend on whether you were asked the question in your native language or a foreign tongue so long as you understood the problem. And yet here we report evidence that people using a foreign language make substantially more utilitarian decisions when faced with such moral dilemmas. We argue that this stems from the reduced emotional response elicited by the foreign language, consequently reducing the impact of intuitive emotional concerns. In general, we suggest that the increased psychological distance of using a foreign language induces utilitarianism. This shows that moral judgments can be heavily affected by an orthogonal property to moral principles, and importantly, one that is relevant to hundreds of millions of individuals on a daily basis.