WordPress – List all posts by year with pagination

Right now I’m using the following code to display all posts by year:

index.php

<?php foreach(posts_by_year() as $year => $posts) : ?>
  <h2><?php echo $year; ?></h2>

  <ul>
    <?php foreach($posts as $post) : setup_postdata($post); ?>
      <li>
        <a href="<?php the_permalink(); ?>"><?php the_title(); ?></a>
      </li>
    <?php endforeach; ?>
  </ul>
<?php endforeach; ?>

functions.php

function posts_by_year() {
  // array to use for results
  $years = array();

  // get posts from WP
  $posts = get_posts(array(
    'numberposts' => -1,
    'orderby' => 'post_date',
    'order' => 'ASC',
    'post_type' => 'post',
    'post_status' => 'publish'
  ));

  // loop through posts, populating $years arrays
  foreach($posts as $post) {
    $years[date('Y', strtotime($post->post_date))][] = $post;
  }

  // reverse sort by year
  krsort($years);

  return $years;
}

And right now I have set at the reading options in the admin panel that I only want to show 3 posts per page. With this new code, the posts by year, it displays all posts and in the pagination buttons there are 3 pages. When switching to another page, it displays the same content all over again.

How can I include the pagination now that I have posts by year? My pagination code is:

<?php if(function_exists('wpex_pagination')) { wpex_pagination(); } ?>

Which calls the function:

if ( !function_exists( 'wpex_pagination' ) ) {

  function wpex_pagination() {

    $prev_arrow = is_rtl() ? '→' : '←';
    $next_arrow = is_rtl() ? '←' : '→';

    global $wp_query;
    $total = $wp_query->max_num_pages;
    $big = 999999999; // need an unlikely integer
    if( $total > 1 )  {
       if( !$current_page = get_query_var('paged') )
         $current_page = 1;
       if( get_option('permalink_structure') ) {
         $format = 'page/%#%/';
       } else {
         $format = '&paged=%#%';
       }
      echo paginate_links(array(
        'base'      => str_replace( $big, '%#%', esc_url( get_pagenum_link( $big ) ) ),
        'format'    => $format,
        'current'   => max( 1, get_query_var('paged') ),
        'total'     => $total,
        'mid_size'    => 3,
        'type'      => 'list',
        'prev_text'   => $prev_arrow,
        'next_text'   => $next_arrow,
       ) );
    }
  }  
}

The idea is that I’m able to set a number of posts to be displayed. Not by year, but by posts. So if I have 30 from 2014 and 20 from 2013 and have the limit set to 5, it’ll only show the first 5 from 2014.

Thanks in advance!

EDIT: Also found the following code, which works, but only on the first page. Meaning that in the first page it shows the date once, but in the following pages it shows the date for each post.

<?php
    $date = 0;
    $newDate = true;
    if (have_posts()) : while (have_posts()) :

        the_post();

    if ($date == 0)
        $date = the_date('Y');
    else if ($date != the_date('Y')) {
        $date = the_date('Y');
        $newDate = true;
    }

    if ($newDate)
        echo $date . ' ';

    $newDate = false; ?>
    <?php get_template_part('content'); ?> 
<?php endwhile; endif; ?>