I am very new to both Monads and Monoids and recently also learned about MonadPlus. From what I see, Monoid and MonadPlus both provide a type with a associative binary operation and an identity. (I'd call this a semigroup in mathematical parlance.) So what is the difference between Monoid and MonadPlus?
A semigroup is a structure equipped with an associative binary operation. A monoid is a semigroup with an identity element for the binary operation.
Every monad has to adhere to the monad laws. For our case, the important one is the associativity law. Expressed using >>=:
(m >>= f) >>= g ≡ m >>= (\x -> f x >>= g)
Now let's apply this law to deduce the associativity for >> :: m a -> m b -> m b:
(m >> n) >> p ≡ (m >>= \_ -> n) >>= \_ -> p
≡ m >>= (\x -> (\_ -> n) x >>= \_ -> p)
≡ m >>= (\x -> n >>= \_ -> p)
≡ m >>= (\x -> n >> p)
≡ m >> (n >> p)
(where we picked x so that it doesn't appear in m, n or p).
If we specialize >> to the type m a -> m a -> m a (substituting b for a), we see that for any type a the operation >> forms a semigroup on m a. Since it's true for any a, we get a class of semigroups indexed by a. However, they are not monoids in general - we don't have an identity element for >>.
MonadPlus adds two more operations, mplus and mzero. MonadPlus laws state explicitly that mplus and mzero must form a monoid on m a for an arbitrary a. So again, we get a class of monoids indexed by a.
Note the difference between MonadPlus and Monoid: Monoid says that some single type satisfies the monoidal rules, while MonadPlus says that for all possible a the type m a satisfies the monoidal laws. This is a much stronger condition.
So a MonadPlus instance forms two different algebraic structures: A class of semigroups with >> and a class of monoids with mplus and mzero. (This is not something uncommon, for example the set of natural numbers greater than zero {1,2,...} forms a semigroup with + and a monoid with × and 1.)