Benchmarks: Add Mixture of Experts Model

[dpower4]

Added MoE model using MixtralConfig.

1. Added 8x7b and 8x22b variants
2. Requires high VRAM as all experts are loaded in memory. Thus, disabled training due to memory constraint on test worker.

[codecov]

Codecov Report

Attention: Patch coverage is 88.05970% with 16 lines in your changes missing coverage. Please review.

Project coverage is 85.65%. Comparing base (249e21c) to head (83c9981).

Files with missing lines	Patch %	Lines
...nch/benchmarks/model_benchmarks/pytorch_mixtral.py	86.44%	16 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #679      +/-   ##
==========================================
+ Coverage   85.61%   85.65%   +0.04%     
==========================================
  Files          99      100       +1     
  Lines        7165     7299     +134     
==========================================
+ Hits         6134     6252     +118     
- Misses       1031     1047      +16     

Flag	Coverage Δ
cpu-python3.10-unit-test	70.63% <39.84%> (-1.21%)	⬇️
cpu-python3.7-unit-test	70.00% <7.46%> (-1.81%)	⬇️
cpu-python3.8-unit-test	70.66% <40.45%> (-1.17%)	⬇️
cuda-unit-test	83.49% <87.21%> (+0.11%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.