For what Alt in Monoid instance needed?

In Monoid and Semigroup instances of Alternative Alt used.

Why we can't write instance without it?

{-# LANGUAGE FlexibleInstances #-}
instance Alternative f => Semigroup (f a) where
  (<>) = <|>

instance Alternative f => Monoid (f a) where
  mempty = empty

And if we can write that, can we then replace Alternative with (Monoid (f a), Applicative f) in functions?

You use it for deriving Monoid for any Alternative

{-# Language DerivingVia #-}

data F a = ..
  deriving (Semigroup, Monoid)
  via Alt F a

instance Functor     F where ..
instance Applicative F where ..
instance Alternative F where ..

Alt is a newtype for good reason as there are many ways to describe Monoid behaviour for an applied type f a. For example Applicative lifting: Ap.

{-# Language DerivingVia #-}

data G a = ..
  deriving (Semigroup, Monoid, Num, Bounded)
  via Ap G a

instance Functor     G where ..
instance Applicative G where ..

The instances you give are maximally overlapping, the Monoid instance of any applied type is now forced to be the Alternative instance, completely ignoring the a parameter.

There are many instances where this would not be correct, for example Semigroup a => Semigroup (Maybe a) is not the same as the Semigroup you would get from Alternative Maybe.

It is possible using a rather new feature QuantifiedConstraints to quantify over the argument of a type constructor forall x. Monoid (f x). This is not the same as Alternative but similar

{-# Language QuantifiedConstraints #-}
..

type Alternative' :: (Type -> Type) -> Constraint

class    (forall x. Monoid (f x)) => Alternative' f
instance (forall x. Monoid (f x)) => Alternative' f